LintLLM
Benchmark for LLM-based Verilog defect detection built with mutation-injected faults, measuring correct detection rate and false-positive rate against traditional EDA lint tools.
What it does and how it connects
LintLLM is an open-source Verilog linting framework based on large language models, built to detect potential defects in Verilog RTL with higher accuracy and lower false positives than traditional EDA tools. It introduces a Prompt of Logic-Tree template to guide LLM reasoning and a Defect Tracker that identifies root defects behind multiple secondary issues, cutting redundancy in the output.
Its benchmark holds 90 Verilog modules across 3 difficulty levels, with 11 categories of mutation-injected defects, 30 modules each at simple, medium and complex. The paper's results table reports o1-mini with LintLLM at 83.33 percent correct detection and 12.22 percent false positives, against 64.44 percent and 27.78 percent for a commercial EDA tool and 62.22 percent and 32.22 percent for Verilator.
The work was published at GLSVLSI 2025. The README states the project is open-sourced under the MIT License, but no LICENSE file exists in the repository root, so the license here rests on the README statement alone.
Alternatives
Compare side by side- RTL-BenchLS
A 10,028-design benchmark for RTL reasoning and generation, rather than a linting framework for defect detection.
- GenAI EDA Workflows Bench
A 50-task RTL generation platform, rather than a linting framework for defect detection.
- AutoBench
Grades LLM-generated Verilog testbenches, rather than a linting framework for defect detection.
- PartForge evals
An eval suite for verification workflows, rather than a Verilog linting framework.
- Rapidflare suite
Rapidflare's internal eval practice for AI agents in electronics, rather than a Verilog linting framework.
Sources
Not verified
- Pricing: none stated on the repo.
- Real user voices: no real linked post read for this page; the repo has no public issues.
- License: the README states MIT, but no LICENSE file exists in the repository root, so the license rests on the README statement alone.
- No GitHub releases exist, so latest release fields are null.
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.