CircuitReason-1K

CircuitReason-1K is a benchmark of 1,000 authentic textbook problems for evaluating long-horizon visual-to-symbolic reasoning in electrical circuits. Each problem pairs one or more circuit diagrams with a self-contained question, a typed or semantically specified answer, and a reference worked solution.

A solver must ground symbols and labels, recover latent topology, select a physical model, formulate coupled equations, propagate intermediate quantities, and preserve units, signs, directions, and phase conventions. An evidence-first construction pipeline aligns questions, figures, and solutions, and a reasoning-oriented taxonomy organizes problems by circuit type and dependency depth.

Evaluation combines conservative typed scoring with identity-blinded multi-model semantic consensus, keeping every problem in the denominator. Across three commercial chatbot systems and six open-source multimodal models, the highest-scoring system reaches 84.8 percent accuracy, while performance consistently deteriorates on long-horizon problems.