MARB
Mechanical Assembly Readiness Benchmark: can an AI assemble a real machine.
What it does and how it connects
MARB is a tool-independent, automatically graded benchmark that measures how correctly AI-assisted CAD places the parts of a real machine: the right position, the right orientation, and the right gap at every interface. The model is given only the authored parts and a set of goal renders (a 3/4 isometric overview plus front, top, and side views) and receives no build steps. Any tool or agent can be tested, including Autodesk Fusion, CadQuery, or a custom agent.
It grades three positional metrics against the answer key under a fixed standard: GAP, the error between actual and intended interface gap, reported as a median in millimeters and the primary functional score; ORIENT, the share of parts placed in the correct rotation; and POS, the position error of each part versus the answer key after best-fit rigid alignment. Buildability stays on as a secondary gate.
MARB is developed by Sunnyday Technologies and runs on the open-source CADCLAW assembly and verification framework, with MARB's graders importing CADCLAW's gates. Grades, run registry, token ledger, and method are all open, and the project home is marb.cadclaw.io. An experimental FEA analysis exists separately from the scores, and the README states full structural-task qualification and physical validation remain incomplete.
Alternatives
Compare side by side- cadclamp
A benchmark that scores AI-generated CAD on whether a single part can be printed, rather than on assembly placement of a real machine.
- deepee
A benchmark for coding agents on embedded firmware, code repair, schematics, and PCB work, rather than on mechanical assembly.
- HW-Benchmarker
A benchmark for RTL debugging on real open-source cores, rather than for mechanical assembly.
- rtlscout-bench
A benchmark suite for LLM agents pushing hardware designs through synthesis, rather than for mechanical assembly.
- Text2CAD (MLYengineering)
A generator that emits STEP and FCStd artifacts from natural language, rather than a benchmark that grades assembly placement.
Sources
Not verified
- Pricing: none stated on the repo.
- No GitHub releases exist, so latest release fields are null.
- Real user voices: the repo's only open item is a pull request by a contributor, and no independent user post was found.
- Related Error Index entries: no matching entry was verified.
- A named individual builder beyond the Sunnyday Technologies organization
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.