CADEngBench
Two-track engineering CAD benchmark covering 300 parametric parts with zero-to-CAD and functional-editing tasks scored on B-Rep validity, DFM checks, parameter perturbations, and matched CalculiX FEA, plus 150 body pairs tested on joint grounding and kinematic verification.
What it does and how it connects
CADEngBench is a two-track benchmark that tests whether CAD output is engineering-grade rather than merely plausible-looking. CADEngBench-P covers 300 parametric parts, each with one zero-to-CAD task and one functional-editing task, scored on B-Rep validity, engineering and DFM checks, parameter-family perturbations, functional editing and matched linear-static FEA in CalculiX. CADEngBench-A covers 150 body pairs through ranked joint retrieval, exact face-and-edge grounding, joint-frame prediction and kinematic verification.
Across eight multimodal, code-capable models, the paper finds editing supplied CAD substantially easier than generating it, while complex edits and matched FEA remain difficult, and assembly predictions often locate the relevant region but fail to recover the recorded joint or mating entities.
The public repository is an anonymous review supplement, not the full corpora: it ships ten development items for the P track and seven for the A track, with the runners, evaluators, schemas and setup scripts, and notes that full-paper aggregates require the sealed full corpora. The P evaluator and runner code is identified as Apache-2.0 in docs/LICENSES.md, while the A handoff carries no standalone license and the derived sample data carries its own notices.
Alternatives
Compare side by side- Engine Bench 0.1
Two models race head to head to build one engine from one prompt, rather than a two-track benchmark of parametric parts and assemblies.
- COSMO-Agent
A closed-loop CAD-CAE framework with its own executable task benchmark, rather than a parametric design and assembly benchmark.
- cad-silent-failure-bench
Measures silent failures where CAD code runs but the part is wrong, rather than scoring engineering behavior across parts and assemblies.
- P3D-Bench
A CAD benchmark scored on geometry and part coverage, rather than one scored on B-Rep validity, DFM and matched FEA.
- StepScore CAD Benchmark
A 44-metric CAD benchmark scored against human-calibrated thresholds, rather than one scored on B-Rep validity and CalculiX FEA.
Sources
Not verified
- Pricing: none stated on the repo or paper page.
- Real user voices: no real linked post read for this page; the repo has no public issues.
- No GitHub releases exist, so latest release fields are null.
- License: no LICENSE file exists in the repo; the Apache-2.0 identification for the P evaluator code comes from docs/LICENSES.md, and the A handoff has no standalone license.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.