Self-serve · $19 once
Stress-test pack
2,400 labelled off-road frames in dust, night, fog, rain and mud (3 severities each), plus real rain. Run your model on them, run our script, and see where it breaks — in about 10 minutes.
See the pack →Stress-test pack · GOOSE edition · v1
Find out where your off-road segmentation breaks — for $19.
150 real off-road frames, each in dust, night, fog, rain on the lens and mud on the lens at three severities, plus 60 real rain and snow frames. All labelled. Run your model, run one script, get your failure map.
How it works
- Download the pack (≈ 225 MB): images, pixel labels, a manifest and
evaluate.py. - Predict with your model — one PNG of class ids per image. A 10-line loop is in the README.
- Score:
python evaluate.py --pred your_predictionsprints mIoU per condition and severity, the drop against clear weather, traversable-ground IoU and the classes that fail first.
Different class set? Map yours to the 19 classes first, or leave out the ones you don’t have. Pixels a dense dust or fog veil makes physically invisible are marked void and never scored — no penalty for not seeing the invisible.
$19one-time · instant download after payment
Buy the pack — $19 →Secure checkout by Lemon Squeezy (card, Apple Pay, Google Pay, PayPal). Download link right after payment.
Free sample · 10 frames
Try the format first
Same structure, same evaluate.py, 10 frames in every condition.
Download siltframe-sample-stress-test.zip ↓What the failure map looks like
Our own reference models on the pack. Yours comes out in the same format.
| Reference model | Clean mIoU | Dust | Night | Fog | Rain on lens | Mud on lens | Real rain / snow |
|---|---|---|---|---|---|---|---|
| SegFormer-B0, RELLIS-3D + GOOSE train | 52.5 | −43% | −61% | −20% | −41% | −20% | −12% |
| Mask2Former Swin-T, RELLIS-3D + GOOSE train | 58.1 | −35% | −56% | −16% | −41% | −19% | −8% |
| SegFormer-B0, RELLIS-3D only | 14.5 | −43% | −69% | −23% | −23% | −10% | −3% |
Relative mIoU drop vs. clean frames, mean of three severities — exactly what evaluate.py prints for your model. None of these models saw these frames.
Honest limits
- The degradations are synthetic and physically motivated; they find weaknesses early and cheaply. Confirm fixes on real frames from your own conditions — the 60 real rain/snow frames are a first check.
- Frames are German off-road scenes at 819×400. If your terrain or camera is very different, an audit or your own frames will tell you more.
- Licence: derived from the GOOSE dataset (Fraunhofer IOSB), CC BY-SA 4.0 — commercial use allowed with attribution; shared derivatives stay CC BY-SA.
Need more than a self-test?
Done for you · from $300
Robustness audit
We test your model on your frames, find why it fails — like tree canopy read as sky in our own case study — and hand you a report with a ranked fix list: which real data to capture, what to synthesise.
Sample report →Done for you · from $300
Hard cases on your data
Your own labelled frames rendered in the conditions you can’t easily capture — dust, night, rain, mud, fog — with labels kept pixel-exact.
Send a brief →