Engine Bench 0.1
Head-to-head agentic CAD benchmark where two models race to build a 1990s 1.6 L inline-four from one prompt and six identical CAD tools, scored 0 to 100 against an 894-part reference engine on silhouette, surface match, key dimensions, and part coverage, with a public leaderboard of replayable runs.
What it does and how it connects
Engine Bench 0.1 is a head-to-head agentic CAD benchmark in which two models race to build a 1990s 1.6 L inline-four from one prompt and six identical CAD tools, scored 0 to 100 against an 894-part reference engine. Everything runs in the browser: each model gets its own workspace, its own API key goes only from the browser to its provider, and ToolsEnabled receives nothing from a run.
Both agents get one fixed system prompt and spec pack and the same six tools: add_parts, remove_parts, list_parts, render_view, get_reference and finish. A safety stop at 150 turns or 90 minutes guards against a runaway loop, and the run ends when the agent calls finish or stops calling tools.
Scoring weights silhouette at 25 percent, surface match at 20 percent, key dimensions at 35 percent and part coverage across 18 named roles at 20 percent. Run records are replayable: checking a record re-animates the recorded CAD calls and re-checks the total within 0.5 points without calling a model, and the leaderboard is edited by hand from verified records.
Alternatives
Compare side by side- CADEngBench
A two-track benchmark of parametric parts and assemblies scored on B-Rep validity and matched FEA, rather than a head-to-head race to build one engine.
- COSMO-Agent
A closed-loop CAD-CAE framework with about 20,000 tasks, rather than a head-to-head race to build one engine.
- cad-silent-failure-bench
Measures silent failures where CAD code runs but the part is wrong, rather than a head-to-head race to build one engine.
- P3D-Bench
A CAD benchmark scored on geometry and part coverage, rather than a head-to-head race to build one engine.
- StepScore CAD Benchmark
A 44-metric CAD benchmark scored against human-calibrated thresholds, rather than a head-to-head race to build one engine.
Sources
Not verified
- Pricing: none stated on the repo or benchmark page.
- Real user voices: no real linked post read for this page; the repo has no public issues.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.