GenAI EDA Workflows Bench
Reproducible 50-task platform for Verilog RTL generation across combinational, sequential, FSM, and mixed designs, validated through a five-phase pipeline of constrained prompting, waveform analysis, formal equivalence checking, and AST-based repair, scored on lint, simulation, and synthesis.
What it does and how it connects
This is a reproducible 50-task platform for Verilog RTL generation across 23 combinational, 14 sequential, 8 FSM and 5 mixed designs, evaluated through a five-phase pipeline of constrained prompting, waveform analysis, formal equivalence checking and AST-based repair, and scored on lint, simulation and synthesis.
Three open-source models were evaluated through Ollama: Llama 3 8B Instruct, StarCoder2 7B and TinyLlama 1.1B. Every run computes syntax validity, functional correctness, generation time and compile and simulation time, and phase 4 and later runs add iteration counts, confidence entropy, waveform diff summaries, formal equivalence status and semantic repair records.
The README is candid about what was planned but not implemented: synthesis quality, testbench detection rate, prompt sensitivity, hallucination index and usability score were never computed in any benchmark run, with the SynthesisTool class existing in code but never invoked. All 12 benchmarks, 1,610 runs, were conducted locally on macOS, and the Docker setup was validated by running the mini benchmark end to end in the container. Code is MIT and the dataset is CC BY-NC 4.0.
Alternatives
Compare side by side- RTL-BenchLS
A 10,028-design benchmark scored by formal equivalence, rather than a 50-task platform scored on lint, simulation and synthesis.
- LintLLM
A benchmark for LLM-based Verilog defect detection, rather than a 50-task RTL generation platform.
- AutoBench
Grades LLM-generated Verilog testbenches, rather than grading RTL generation itself.
- PartForge evals
An eval suite for verification workflows, rather than a 50-task Verilog generation platform.
- Rapidflare suite
Rapidflare's internal eval practice for AI agents in electronics, rather than a 50-task Verilog generation platform.
Sources
Not verified
- Pricing: none stated on the repo.
- Real user voices: no real linked post read for this page; the repo has no public issues.
- No GitHub releases exist, so latest release fields are null.
- Dataset license is CC BY-NC 4.0, which is non-commercial; several metrics the README defines were never computed.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.