CLOSER-Bench
Budgeted cross stage design closure protocol pairing spec to RTL, RTL to GDS and spec to GDS tasks for one design and one hidden objective, recording every simulator, synthesis, STA and place and route call and scoring final quality, anytime progress, tool cost and cross stage recovery on open source Verilator, Yosys, OpenROAD, KLayout and Sky130.
What it does and how it connects
CLOSER-Bench, short for Cross-Layer Objective-driven Stage-paired Evaluation under Resource constraints, is a controlled evaluation protocol for budgeted cross-stage design closure. For one design and one hidden objective it pairs spec-to-RTL, RTL-to-GDS and spec-to-GDS tasks, records every simulator, synthesis, STA and place-and-route invocation, and measures final quality, anytime progress, tool cost and cross-stage recovery.
The protocol is built on open-source Verilator, Yosys, OpenROAD, KLayout and Sky130, plus the Harbor agent harness, and a shared AXI/DMA streaming design with an SRAM macro and executable functional oracle. The evaluator computes a validity gate over functional, LEC, gate-level simulation, STA, route, DRC and LVS factors, and scores against pre-registered area, delay, power and cycle targets. Budgets are tiered: low, medium and high, with up to 3, 8 or 20 full place-and-route calls and 4, 10 or 24 hour wall-clock limits.
A ten-task pilot spanning RTL repair, mutation-based verification, coverage, PPA optimization, design-space exploration, cross-model debugging and security establishes the executable harness and exposes a sharp completion-closure gap: three agents solve a localized AXI repair task, while the matched verification-closure task separates a frontier agent from two otherwise successful baselines. The paper reports the completed pilot evidence and treats the full three-way agent matrix as future evaluation rather than estimating unobserved results.
Alternatives
Compare side by side- HSCO-Bench
An end-to-end hardware-software co-design benchmark deploying SoC prototypes on an FPGA, rather than a budgeted cross-stage closure protocol.
- ArchXBench
A hierarchical RTL synthesis benchmark from simple gates to full subsystems, rather than a budgeted cross-stage closure protocol.
- AgenticBBO-Bench
A cross-domain benchmark for LLM agents doing black-box optimization under a fixed budget, rather than a budgeted cross-stage hardware closure protocol.
- Phoenix-bench
511 verified Verilator instances across 114 hardware repos with Docker-pinned EDA toolchains, rather than a budgeted cross-stage closure protocol.
Recurring errors it addresses
Error Index entries this tool is built to reduce or work around.
- The synthesis check re-hashes RTL and liberty but records no hash of the netlist it produced, so it cannot catch a committed gate-level netlist that was hand-edited or went stale after resynthesis. It also needs the PDK on the checking host, so signoff CI skips it.Error Index
- agent wrappers for commercial RTL-to-GDS tools shipped with two command names that were not real binaries (conformal is lec, starrc is StarXtract), a timeout left a pt_shell child holding a licence, and result markers come from documented log formats rather than real logs, with unmeasured parts labeled pending.Error Index
Sources
Not verified
- License: no license is stated on the arXiv page read, so no license is recorded; the catalog label source-available is unverified.
- Pricing: none stated on the pages read.
- No independent user voice found: the source read is the paper.
- A public code repository: none is linked in the paper read; the paper text contains no GitHub link.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.