LFORLA FreeCAD Fix
Agents repair broken parametric FreeCAD scripts, graded by deterministic geometry-oracle scoring on a public leaderboard.
What it does and how it connects
The FreeCAD Fix benchmark on LFORLA diagnoses and repairs a broken parametric FreeCAD script. Every model receives the identical broken script and the identical repair instructions, so the comparison is apples to apples. The benchmark spans CAD, engineering, code and reasoning, with a rubric per axis.
Scoring is by a deterministic geometry oracle, not an LLM judge. The repaired script is executed with freecadcmd and the resulting geometry is measured against the expected geometry, so the same repaired script always produces the same score. A score of 100 means the repaired script passed every geometry check.
Every answer is stored verbatim on a public trail, so anyone can re-read what a model actually answered rather than only its final score. The LFORLA team's writeup reports leaderboard scores including Mimo v2.6 Flash at 100 and Qwen 3.8 27B at 99.33, and cautions that the leaderboard is a snapshot and a high score on this one repair task does not mean a model is good at general reasoning.
Alternatives
Compare side by side- Parametric CAD Bench (gNucleus)
Agents author editable FreeCAD Part Design models from a natural-language spec, scored by a harmonic mean of geometry and spec consistency, rather than repairing a broken parametric FreeCAD script.
- BenchCAD
Scores execution-verified CadQuery programs by deterministic voxel IoU across 106 industrial part families, rather than repairing broken FreeCAD scripts with a geometry oracle.
- MUSE
Grades text-to-CAD assemblies in a three-stage funnel ending in a VLM judge, rather than repairing a broken FreeCAD script against a deterministic geometry oracle.
- Text2CAD-Bench
Tests text-to-parametric-CAD on 600 human-curated examples across four complexity levels, rather than repairing a broken parametric FreeCAD script.
Recurring errors it addresses
Error Index entries this tool is built to reduce or work around.
- When an agent fans out parallel create calls through the FreeCAD MCP bridge, the shared connection races: the first object is created even though its call reports failure, and the second object is silently never created with the status still showing healthy.Error Index
- A FreeCAD MCP bridge silently ignored parameters the client materialized from JSON-Schema defaults, so box(size=20) produced a default 10mm shape. The agent believes it built the requested geometry while the model is something else entirely.Error Index
- A FreeCAD MCP bridge's autosave-before-call rewrote the active document on every tool call even with zero unsaved changes, stamping phantom diffs onto git-tracked CAD files. Agent activity pollutes version control with edits no human authored.Error Index
- A FreeCAD MCP bridge's persistent execution namespace meant that once any call bound a variable named result, every later call echoed the old value as its own output. The agent reads that output and believes its code produced a value it never computed.Error Index
- A FreeCAD user reports fillet/bevel as big weaknesses of FreeCAD and shows a fillet that breaks in FreeCAD and in the FreeCAD-Nxt add-on meant to fix it.Error Index
Sources
Not verified
- Pricing: no price is stated on the LFORLA homepage read.
- License terms: no terms page was read; commercial is the directory record's classification.
- A public repo: none found; the benchmark runs on the LFORLA platform.
- Real user voices: no real linked post read for this page. The methodology writeup cited is published by the LFORLA team itself, so it is a vendor source, not a user voice.
- Related Error Index entries: the FreeCAD entries below are about FreeCAD agent bridges and FreeCAD fillet behavior, not about this benchmark's own scoring.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.