CADBench (BlenderLLM)
500 simulated plus 200 forum samples testing LLM generation of Blender CAD scripts against multi-dimensional criteria.
What it does and how it connects
CADBench is the benchmark inside the BlenderLLM repository for evaluating LLM generation of CAD scripts. It contains 500 simulated data samples and 200 data samples collected from online forums, each assessed on multi-dimensional criteria covering attributes, spatial relations and instruction following, with a syntax error rate reported separately. GPT-4o is used to evaluate models on the benchmark.
BlenderLLM itself is a large language model fine-tuned on BlendNet, a dataset of 12K instruction and CAD-script pairs, based on Qwen2.5-Coder-7B-Instruct. The generated scripts are executed in Blender to render 3D models; Blender is required for execution and rendering, while chat alone works without it.
On CADBench, BlenderLLM averages 0.748 on the simulated split and 0.664 on the forum split, ahead of the baselines in the README table, which include o1-Preview, GPT-4-Turbo and Claude-3.5-Sonnet. The README states three limitations: basic modeling only, with no material properties, surface treatments or internal structural details; text-only input with no multimodal input; and no multi-turn dialogue.
Alternatives
Compare side by side- BenchCAD
Scores execution-verified CadQuery programs by deterministic voxel IoU across 106 industrial part families, rather than Blender CAD scripts judged on multi-dimensional criteria.
- Text2CAD
A framework and dataset of around 170K models that generates sketch-and-extrude sequences from text, rather than a benchmark for Blender CAD script generation.
- MUSE
Grades text-to-CAD assemblies in a three-stage funnel ending in a VLM judge, rather than benchmarking Blender CAD script generation with a GPT-4o evaluator.
- CADBench (Doris et al.)
A unified 18,000-sample benchmark turning mesh, image or render inputs into STEP or mesh, rather than a Blender script generation benchmark.
- DeepCAD (dataset)
A dataset of 178K sketch-and-extrude sequences parsed from Onshape, rather than a Blender CAD script generation benchmark.
Sources
Not verified
- Pricing: none stated on the repo (open source model and benchmark).
- Real user voices: no independent user report found.
- No GitHub releases exist, so latest release fields are null.
- Related Error Index entries: no matching entry verified (the live error map has no Blender entries).
- A named individual builder beyond the FreedomIntelligence organization.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.