DocsDietBench v0
Docs
DietBench v0
DietBench v0 is a 500-query corpus for evaluating dietary restaurant search behavior.
Corpus status
DietBench v0 is generated and unlabeled. Labeling is still in progress, so benchmark results are coming soon. Compass will not publish win claims until labeled evaluation is complete.
Corpus stats
| Field | Type | Notes |
|---|---|---|
| Total queries | 500 | Synthetic JSONL records generated with seed 20260501. |
| Intent categories | 7 | Strict vegan, flexible vegan, contamination, safety-sensitive, intersection, edge case, multilingual. |
| Complexity split | 200 / 200 / 100 | Simple, medium, and complex queries. |
| Primary language | English | 480 English records plus 20 multilingual records. |
| Labels | 0 complete | Manual labeling is the next step. |
Category distribution
| Field | Type | Notes |
|---|---|---|
| strict_vegan | 150 | Strict vegan search intent. |
| vegan_flexible | 80 | Flexible vegan and vegan-options intent. |
| cross_contamination | 80 | Shared fryer, grill, surface, and prep concerns. |
| safety_sensitive_vegan | 70 | Vegan plus safety-sensitive constraints. |
| dietary_intersection | 50 | Mixed diet or multi-constraint queries. |
| edge_case | 50 | Ambiguous or low-evidence cases. |
| multilingual | 20 | Non-English or mixed-language queries. |