RealCADBench

RealCADBench is a benchmark for intent-to-program CAD modeling from real industrial design intents. It contains 12,632 tasks from 19 factory-automation categories, spanning text descriptions, 2D engineering drawings, real product pictures, and rendered images, covering both part and assembly modeling.

Each method generates FreeCAD API Python, which a shared runtime executes to export the 3D model. The paper reports results on a 1,770-task evaluation slice: 1,745 part tasks across four input regimes and RCB-Assm25, a 25-task assembly study used in every reported assembly comparison. The exported model is evaluated with executability, Solid IoU, Surface IoU, and a rubric-based visual-semantic identity Judge.

Among the nine standalone frontier large models evaluated, no model leads all four metrics. Across the six frontier-scale large models, executability ranges from 0.565 to 0.812, Solid IoU from 0.2841 to 0.5379, and Surface IoU from 0.112 to 0.217 across the four part regimes. On RCB-Assm25, Codex with GPT-5.5 improves executability and both IoU metrics over standalone GPT-5.5 but lowers the Judge by 6.98 percentage points, leaving GPT-5.5 as the Judge leader. Recurring failure modes observed are missing fine structures, loss of part identity, and incorrect assembly placement.