MUSE
Grades NL-to-CadQuery generation in a 3-stage funnel (execute, geometry, VLM judge) on functionality, manufacturability, and assemblability.
What it does and how it connects
MUSE is a text-to-CAD benchmark focused on complex, editable boundary representation (B-Rep) assemblies, rather than isolated primitive models. It pairs practical design instances with structured Design Specifications and evaluates generated models for engineering usability, not only geometric similarity.
Evaluation runs through a three-stage funnel where one failed stage means downstream zero. Stage 1 is a code check: does the generated CadQuery script execute in a sandbox. Stage 2 is a geometry check with four binary OCCT checks: watertight, manifold, self-intersection free and overlap free. Stage 3 is design-intent alignment, where a rubric-based visual language model judge (Gemini-3.1-Pro) scores functionality, manufacturability and assemblability against a per-case rubric, with the final score as the mean.
The project page reports a failure cascade from executable code to valid geometry to engineering-ready design, with even the strongest models achieving limited success on fine-grained engineering criteria. The leaderboard page lists 15-plus models across 10 metrics.
Alternatives
Compare side by side- BenchCAD
Scores execution-verified CadQuery programs by deterministic voxel IoU with no LLM judge, rather than grading through a three-stage funnel ending in a VLM judge.
- Parametric CAD Bench (gNucleus)
Agents author editable FreeCAD Part Design models in a multi-step loop, scored by a harmonic mean of geometry and spec consistency, rather than single-shot CadQuery generation judged in three stages.
- Text2CAD-Bench
Tests text-to-parametric-CAD on 600 human-curated examples across four complexity levels, rather than evaluating editable B-Rep assemblies through a code, geometry and design-intent funnel.
- CADBench (Doris et al.)
A unified 18,000-sample benchmark turning mesh, image or render inputs into STEP or mesh, rather than a funnel that grades functionality, manufacturability and assemblability of B-Rep assemblies.
- DeepCAD (dataset)
A dataset of 178K sketch-and-extrude sequences parsed from Onshape that text-to-CAD benchmarks build on, rather than a funnel grading B-Rep assemblies for engineering usability.
Sources
Not verified
- Pricing: none stated on the project page (research benchmark).
- Real user voices: no real linked post read for this page.
- Related Error Index entries: no matching entry verified.
- The number of design instances in the benchmark: the project page read does not state a count.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.