MMMU Mechanical Engineering
Multiple-choice questions in mechanical engineering, registered as subject evals in OpenAI's open-source evals framework; a general knowledge check, not a design benchmark.
Summary rewritten from the vendor site because the catalog version did not match it.
What it does and how it connects
MMMU Mechanical Engineering is the mechanical-engineering subject of MMMU, as registered in OpenAI's open-source evals framework. The registry file evals/registry/evals/mmmu.yaml defines mmmu-mechanical-engineering dev and validation evals that load the Mechanical Engineering split of the MMMU dataset from Hugging Face and score by accuracy.
The evals framework provides the registry, the runner, and the ability to write custom evals, with the registry data stored using Git LFS. Running evals needs an OpenAI API key and carries API costs, per the README. The MMMU subject is a general knowledge check: it tests understanding through multiple-choice questions with images, not CAD modeling or design work.
This entry covers only the mechanical-engineering subject, not the full MMMU benchmark, and the repository statistics shown are for the whole evals registry.
Stats are for the openai/evals registry repository as a whole, not for the MMMU subject alone.
Alternatives
Compare side by side- DepthBenchCAD
DepthBenchCAD audits how deeply CAD generations are checked, while MMMU Mechanical Engineering is a general multiple-choice knowledge check.
- cadgenbench
CADGenBench scores AI-generated and edited 3D parts against reference geometry, while MMMU Mechanical Engineering is a general multiple-choice knowledge check.
- RLE-Bench
RLE-Bench tests coding agents as robot-learning engineers in simulation, while MMMU Mechanical Engineering is a general multiple-choice knowledge check.
- DFM-Redesign
DFM-Redesign benchmarks intent-preserving CAD redesign for manufacturability, while MMMU Mechanical Engineering is a general multiple-choice knowledge check.
Sources
Not verified
- License: the repository API record lists no license
- Pricing: none stated on the repo (running evals uses a paid OpenAI API key, per the README)
- Real user voices (no real linked post read for this page)
- Related Error Index entries (no matching entry verified)
- The catalog's 463-eval registry figure (not confirmed in the sources read)
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.