EEE-Bench
2,860 hand-curated multimodal problems across 10 EE subdomains; models average 19.48% to 46.78%, a sobering calibration of practical EE skill.
Summary rewritten from the vendor site because the catalog version did not match it.
What it does and how it connects
EEE-Bench is a multimodal benchmark for assessing large multimodal models on practical electrical and electronics engineering tasks. It consists of 2,860 hand-picked and carefully curated multiple-choice and free-form problems spanning 10 essential subdomains, including analog circuits, control systems, digital logic, circuit theory, power electronics, signals and systems, communication systems, electronic devices, electrical machines, and electromagnetics.
Questions are collected from official EE exams and verified online sources in a multimodal setting, with diverse visual contexts such as electric and digital circuits and system diagrams. The paper evaluates 17 leading open-source and closed-source foundation models with 7 popular prompting techniques.
Average model performance ranges from 19.48 percent to 46.78 percent, and the paper notes a tendency toward laziness in overlooking essential visual context. The benchmark was accepted to CVPR 2025. The repository itself holds only a two-line README pointing to the paper, so the paper is the substantive source for this entry.
Alternatives
Compare side by side- MMCircuitEval
MMCircuitEval tests circuit knowledge across EDA stages with 3,614 Q and A pairs, while EEE-Bench tests EE knowledge with 2,860 problems across 10 subdomains.
- ElectroVizQA
ElectroVizQA tests visual question answering on digital-electronics diagrams, while EEE-Bench tests EE knowledge across 10 subdomains.
- CircuitReason-1K
CircuitReason-1K tests long-horizon visual-to-symbolic circuit reasoning, while EEE-Bench tests EE knowledge across 10 subdomains.
- PCB-QA
PCB-QA asks questions about real boards from their KiCad files, while EEE-Bench tests EE knowledge across 10 subdomains.
- OmniSch
OmniSch tests schematic-to-graph visual reasoning, while EEE-Bench tests EE knowledge across 10 subdomains.
Sources
Not verified
- License: the repository API record lists no license and the README states none
- Pricing: none stated on the repo or paper
- Real user voices (no real linked post read for this page)
- Related Error Index entries (no matching entry verified)
- A dataset download or evaluation script in the repository (the repo README only points to the paper)
- A named individual builder beyond the jike338 repository owner
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.