MMCircuitEval
3,614 expert-reviewed multimodal QA pairs across digital and analog circuits, tagged by EDA stage, ability, and difficulty.
What it does and how it connects
MMCircuitEval is a multimodal benchmark for evaluating large language models across electronic design automation tasks. It comprises 3,614 meticulously curated question-answer pairs spanning digital and analog circuits, drawn from textbooks, technical question banks, datasheets, and real-world documentation, with each pair expert-reviewed for accuracy and relevance.
Questions are categorized by EDA stage (general knowledge, specification, front-end design, and back-end design), circuit type, tested ability (knowledge, comprehension, reasoning, computation), and difficulty level. Question types include single-answer choice, multi-answer choice, fill-in-the-blank, and open-ended.
The dataset is available on Hugging Face as charlie314159/MMCircuitEval, loadable by split (general, spec, frontend, backend), and the repository ships a benchmark runner that takes a model and a field as arguments. The paper reports significant performance gaps among existing models, particularly in back-end design and complex computations.
Alternatives
Compare side by side- EEE-Bench
EEE-Bench tests EE knowledge with 2,860 problems across 10 subdomains, while MMCircuitEval tests circuit knowledge across EDA stages.
- ElectroVizQA
ElectroVizQA tests visual question answering on digital-electronics diagrams, while MMCircuitEval tests circuit knowledge across EDA stages.
- PCB-QA
PCB-QA asks questions about real boards from their KiCad files, while MMCircuitEval tests circuit knowledge across EDA stages.
- CircuitReason-1K
CircuitReason-1K tests long-horizon visual-to-symbolic circuit reasoning, while MMCircuitEval tests circuit knowledge across EDA stages.
- OmniSch
OmniSch tests schematic-to-graph visual reasoning, while MMCircuitEval tests circuit knowledge across EDA stages.
Sources
Not verified
- License: the repository API record lists no license and the README states none
- Pricing: none stated on the repo
- Real user voices (no real linked post read for this page)
- Related Error Index entries (no matching entry verified)
- A named individual builder beyond the cure-lab organization
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.