Siltframe Get a fix →

Stress-test pack · GOOSE edition · v1

Find out where your off-road segmentation breaks — for $19.

150 real off-road frames, each in dust, night, fog, rain on the lens and mud on the lens at three severities, plus 60 real rain and snow frames. All labelled. Run your model, run one script, get your failure map.

40-second product demo: the real pack, a real model run on all 2,460 images, the real output of evaluate.py.

How it works

  1. Download the pack (≈ 225 MB): images, pixel labels, a manifest and evaluate.py.
  2. Predict with your model — one PNG of class ids per image. A 10-line loop is in the README.
  3. Score: python evaluate.py --pred your_predictions prints mIoU per condition and severity, the drop against clear weather, traversable-ground IoU and the classes that fail first.

Different class set? Map yours to the 19 classes first, or leave out the ones you don’t have. Pixels a dense dust or fog veil makes physically invisible are marked void and never scored — no penalty for not seeing the invisible.

$19one-time · instant download after payment

We e-mail you a secure payment link within one business day; the download follows right after payment.

Free sample · 10 frames

Try the format first

Same structure, same evaluate.py, 10 frames in every condition.

Or skip the download: run it in your browser — a notebook that scores a model over all 165 frames in seconds. Also on GitHub and Hugging Face, no e-mail, no sign-up. Images and labels CC BY-SA 4.0, evaluate.py MIT.

What the failure map looks like

Our own reference models on the pack. Yours comes out in the same format.

Reference modelClean mIoUDustNightFogRain on lensMud on lensReal rain / snow
SegFormer-B0, RELLIS-3D + GOOSE train52.5−43%−61%−20%−41%−20%−12%
Mask2Former Swin-T, RELLIS-3D + GOOSE train58.1−35%−56%−16%−41%−19%−8%
SegFormer-B0, RELLIS-3D only14.5−43%−69%−23%−23%−10%−3%
Relative mIoU drop vs. clean frames, mean of three severities — exactly what evaluate.py prints for your model. None of these models saw these frames.

Honest limits

  • The degradations are synthetic and physically motivated; they find weaknesses early and cheaply. Confirm fixes on real frames from your own conditions — the 60 real rain/snow frames are a first check.
  • Frames are German off-road scenes at 819×400. If your terrain or camera is very different, an audit or your own frames will tell you more.
  • Licence: derived from the GOOSE dataset (Fraunhofer IOSB), CC BY-SA 4.0 — commercial use allowed with attribution; shared derivatives stay CC BY-SA.

Need more than a self-test?

Individual

$19one-time

Students, researchers, solo builders

Test your model yourself.

  • Stress-test pack: 2,460 labelled off-road images — dust, night, fog, rain and mud on the lens, 3 strengths each, plus real rain and snow
  • Scoring script: your failure map in about 10 minutes
  • Reference results from 3 AI models to compare against
  • + 2 more
Get the pack — $19 →

Team

from $300per audit

Startups and small robotics teams

We find where and why your model fails.

  • Everything in Individual
  • Your model tested on your own frames (up to 300)
  • All 5 conditions × 3 strengths, cross-checked with independent simulators
  • + 3 more
Book an audit →

Recommended

Pro

from $600per model

Teams shipping a product

We find it — and fix it.

  • Everything in Team
  • Gap data: commercially licensed real data that covers what your model never saw, mapped to your classes
  • Simulated hard cases on your own frames — only where real data doesn’t exist
  • + 3 more
Get a fix →

Enterprise

Custom

Fleets, several models or cameras

Robustness as part of every release.

  • Everything in Pro
  • Several models, cameras or operating sites
  • Regression testing on every model release
  • + 3 more
Talk to us →

No hidden costs: every audit ends with a fixed price for the fix, before you commit. Compare plans →