vibe-openscad
Live community leaderboard running tiered OpenSCAD code-generation tasks across dozens of models, recording steps, wall time, token counts, cost, and part renders for side-by-side comparison.
What it does and how it connects
vibe-openscad is a benchmark that measures how well language models handle three dimensional space, using OpenSCAD as the test bed. The same task is given to many models and harnesses, and the public dashboard shows the results side by side: the number of steps taken, wall time, input and output token counts, cost, and a render of the resulting part.
Tasks are defined as YAML in four difficulty tiers: tier 1 basic shapes, tier 2 mechanical parts, tier 3 assemblies, and tier 4 vision tasks that start from a PDF datasheet page. Each task states the prompt and the expected result, such as a bounding box in millimetres and a manifold solid.
Results are shown on two separate axes. Bare runs call a model API once and render whatever SCAD comes back. Harness runs let an agent such as Claude Code iterate through the project's own MCP server, which offers render_openscad and submit_final tools, so the agent can rewrite the code and check its own renders before submitting.
The project is a TypeScript pnpm monorepo with runners and a web dashboard, tested with Vitest. Bench runs are differential, keyed on a signature, so only changed model and task combinations run again. The dashboard is deployed on Cloudflare Workers Static Assets.
Alternatives
Compare side by side- openscad-agent
An agent that writes OpenSCAD code for the user, rather than a leaderboard that scores models on OpenSCAD tasks.
- OpenSuperCAD
A native OpenSCAD editor where an agent writes code and iterates on renders, rather than a benchmark that scores models.
- QymCAD
A parametric B-rep CAD program the user drives directly, rather than a leaderboard for generated OpenSCAD code.
- PartForge evals
A standing CAD-agent evaluation published as model comparisons on the PartForge blog, rather than a live OpenSCAD leaderboard with per-run renders.
Sources
Not verified
- License: no LICENSE file in the repository and the GitHub API returns no license, so no license is stated; the catalog label source-available is not verified.
- Pricing: none stated on the site or repo.
- Real user voices: recent repo activity is the maintainer's own pull requests.
- Related Error Index entries: no OpenSCAD entries exist in the live map.
- A task count: the dashboard shows tiered tasks but no total count is stated on the site or repo, so none is given.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.