HLS-Eval
94 HLS C++ code-generation and code-edit designs with NL descriptions, testbenches, references, parallel eval, and a leaderboard.
What it does and how it connects
HLS-Eval is a benchmark and evaluation framework for LLMs on high-level synthesis (HLS) design tasks, including HLS code generation and HLS code editing. It provides a comprehensive benchmark set of HLS designs with natural-language descriptions, testbenches and reference HLS implementations.
The framework can run evaluations across tasks and models in a parallel and efficient manner. It is implemented in Python and can be easily extended to support new tasks, benchmark designs, models and inference methods. The repository ships a leaderboard directory and installs with uv or pip directly from git.
The work was presented at ICLAD 2025, the IEEE International Conference on LLM-Aided Design, by Stefan Abi-Karam and Cong Hao.
Alternatives
Compare side by side- VerilogEval
156 HDLBits-derived description-to-Verilog tasks verified by Icarus Verilog simulation, rather than HLS C++ code-generation and code-edit tasks.
- RTLLM 2.0
50 realistic natural-language-described designs scored on syntax, functionality, synthesis and PPA, rather than HLS C++ code-generation and code-edit tasks.
- PyHDL-Eval
168 spec-to-RTL problems for Python-embedded HDL DSLs plus Verilog, rather than HLS C++ code-generation and code-edit tasks.
- TuRTLe
A unified harness running VerilogEval, RTLLM, VGen, RTL-Repo and NotSoTiny under several simulators, rather than an HLS benchmark and framework.
- CVDP
783 engineer-authored problems across 13 categories covering RTL design and verification, rather than HLS C++ code-generation and code-edit tasks.
Sources
Not verified
- Pricing: none stated on the repo (open source benchmark).
- Real user voices: no real linked post read for this page.
- No GitHub releases exist, so latest release fields are null.
- Related Error Index entries: no matching entry verified.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.