CADWorld
200 long horizon computer use tasks in FreeCAD across 11 workflow categories from sketching to CAM, FEM and technical drawing, where agents act through screenshots and GUI actions and pass only when task specific executable checks over the saved FreeCAD artifacts succeed.
What it does and how it connects
CADWorld is a benchmark for long-horizon computer use in FreeCAD. It contains 200 tasks spanning 11 mechanical CAD workflow categories: sketching, part modeling, assembly, CAM, FEM, appearance, point cloud, macro, measurement, mesh and technical drawing. Agents operate FreeCAD through screenshots and GUI actions, the same interface a human operator would use, rather than through scripted APIs or programmatic shortcuts.
A pass means the task-specific executable checks passed over the saved FreeCAD artifacts. The project page states a task asks an agent to leave behind a specific engineering result, and that the saved file is part of the answer: geometric checks, feature structure, constraints, CAM outputs and FEM results vary by task. A pass is not a general certification of engineering fitness.
The paper, arXiv 2609.16251, is under review as a conference paper. The project page reports 14 evaluated agents and a best evaluated agent success of 41.5 percent, while the paper reports the strongest of seven agents at 17.5 percent against 87.0 percent for the expert reference; the page labels the higher figure as including extended runs that are not part of the paper. The terminal-only experiment is a separate 50-task ablation and is not pooled into the main results. No public code repository was found: the CAD-WORLD GitHub account holds only the project website repo and a Videos repo.
Alternatives
Compare side by side- BenchCAD 2.0
100 held-out engineering cases where models build CAD parts from drawings and recover board wiring, rather than a computer-use benchmark in FreeCAD.
- RealCADBench
A benchmark for intent-to-program CAD modeling from industrial design intents, rather than a computer-use benchmark in FreeCAD.
- P3D-Bench
A benchmark testing whether multimodal LLMs can generate precise parametric 3D programs and assemblies from text or images, rather than a computer-use benchmark in FreeCAD.
- freecad-mcp (blwfish)
An MCP server exposing FreeCAD modeling, CAM and diagnostics to agents, rather than a benchmark that scores agents inside FreeCAD.
Recurring errors it addresses
Error Index entries this tool is built to reduce or work around.
- When an agent fans out parallel create calls through the FreeCAD MCP bridge, the shared connection races: the first object is created even though its call reports failure, and the second object is silently never created with the status still showing healthy.Error Index
- A FreeCAD MCP bridge silently ignored parameters the client materialized from JSON-Schema defaults, so box(size=20) produced a default 10mm shape. The agent believes it built the requested geometry while the model is something else entirely.Error Index
- A CAD agent bridge's screenshot tool refused any image over a 50KB socket frame, so every real model screenshot was dropped, and images that did fit arrived as base64 text inside JSON instead of real image blocks. The agent's only visual feedback channel was dead for production-scale models.Error Index
- A FreeCAD MCP bridge's autosave-before-call rewrote the active document on every tool call even with zero unsaved changes, stamping phantom diffs onto git-tracked CAD files. Agent activity pollutes version control with edits no human authored.Error Index
Sources
Not verified
- License: no license is stated on the project page or in the paper, so no license is recorded; the catalog label source-available is unverified.
- Pricing: none stated on the pages read.
- No independent user voice found: the sources read are the project page and the paper.
- A public code repository: none was found; the CAD-WORLD GitHub account holds only the website repo.
- Success figures differ between the paper (17.5 percent for the strongest of seven agents) and the project page (41.5 percent best evaluated agent, including extended runs); the entry states both with their scope.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.