ChipNeMo evals (reference)
NVIDIA's domain-adapted LLMs for chip design, evaluated on an engineering-assistant chatbot, EDA script generation, and bug summarization.
Summary rewritten from the vendor site because the catalog version did not match it.
What it does and how it connects
ChipNeMo is NVIDIA's work on domain-adapted large language models for industrial chip design. Instead of deploying off-the-shelf commercial or open-source LLMs directly, it applies domain-adaptive tokenization, domain-adaptive continued pretraining, model alignment with domain-specific instructions, and domain-adapted retrieval models.
It is evaluated on three chip-design LLM applications: an engineering-assistant chatbot, EDA script generation, and bug summarization and analysis. The paper reports that domain-adaptive pretraining leads to superior performance on domain-related downstream tasks compared with the base LLaMA2 counterparts, without degradations in generic capabilities.
The largest model, ChipNeMo-70B, outperforms GPT-4 on two of the three use cases, the engineering-assistant chatbot and EDA script generation, while showing competitive performance on bug summarization and analysis. This entry is a reference only: no public dataset or repo was found, so the evals are citable but not runnable from a public artifact.
Alternatives
Compare side by side- CVDP
783 engineer-authored problems across 13 categories covering RTL design and verification with hidden cocotb harnesses, rather than a domain-adaptation reference with no public dataset or repo.
- VerilogEval
156 HDLBits-derived description-to-Verilog tasks verified by Icarus Verilog simulation, rather than a domain-adaptation reference with no public dataset or repo.
- RTLLM 2.0
50 realistic natural-language-described designs scored on syntax, functionality, synthesis and PPA, rather than a domain-adaptation reference with no public dataset or repo.
- ChipBench
44 hierarchical Verilog modules, 89 injected-bug debug cases and 132 reference-model tasks, rather than a domain-adaptation reference with no public dataset or repo.
- Build-A-Chip-Bench
Full-flow agentic chip design from spec to GDS on Nangate45/OpenROAD, graded on functional verification plus PPA, DRC and signoff, rather than a domain-adaptation reference.
Sources
Not verified
- License: no license is stated on the arXiv page read.
- A public repo or dataset: none; this entry is a reference only.
- Pricing: none stated.
- Real user voices: no real linked post read for this page.
- Related Error Index entries: no matching entry verified.
- The 88-question RAG benchmark detail: directory record; the arXiv abstract read does not state it.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.