RTLLM 2.0
50 realistic natural-language-described designs scored on syntax, functionality, and synthesis plus PPA against a reference design.
Summary rewritten from the vendor site because the catalog version did not match it.
What it does and how it connects
RTLLM is an open-source benchmark for generating design RTL with natural language, from Yao Lu, Shang Liu, Qijun Zhang and Zhiyao Xie (ASP-DAC 2024). Version 2.0 expands the benchmark to 50 designs, meticulously categorized into four classes: Arithmetic, Memory, Control and Miscellaneous modules. Version 2.1, released 2026-08-15, corrected several design descriptions and testbenches.
Each design ships a natural-language description, a testbench, a verified reference Verilog design hand-crafted by human designers, and LLM-generated Verilog. The description states the module name and all input and output signals with names and widths, so functionality can be verified automatically with the provided testbench.
The evaluation workflow has three stages: the LLM generates design RTL from the description, the framework tests functionality with the testbench, and the generated RTL is synthesized into a netlist to compare design quality, including worst negative slack, against the reference design. An automated Python script (auto_run.py) runs one-click compilation across all designs.
Alternatives
Compare side by side- VerilogEval
156 HDLBits-derived description-to-Verilog tasks verified by Icarus Verilog simulation, rather than 50 realistic designs scored on syntax, functionality, synthesis and PPA.
- TuRTLe
A unified harness running VerilogEval, RTLLM, VGen, RTL-Repo and NotSoTiny under several simulators, rather than the RTLLM benchmark alone.
- CVDP
783 engineer-authored problems across 13 categories covering RTL design and verification with hidden cocotb harnesses, rather than 50 natural-language-described designs.
- ChipBench
44 hierarchical Verilog modules, 89 injected-bug debug cases and 132 reference-model tasks, rather than 50 natural-language-described designs.
- RealBench
60 module-level and 4 system-level tasks from real IP, evaluated with Verilator plus JasperGold formal, rather than 50 natural-language-described designs.
Sources
Not verified
- Pricing: none stated on the repo (open source benchmark).
- Real user voices: no real linked post read for this page.
- No GitHub releases exist, so latest release fields are null.
- Yosys and OpenROAD as the synthesis and PPA tools: the directory one-liner named them, but the README read names Synopsys VCS for testbenches and Design Compiler for the paper's PPA figures, so the rewritten one-liner drops the tool names.
- Related Error Index entries: no matching entry verified.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.