RLE-Bench
Nine task families for coding agents as robot-learning engineers, from interactive control and policy learning to perception and mechanical design, graded in physics simulation.
Summary rewritten from the vendor site because the catalog version did not match it.
What it does and how it connects
RLE-Bench is a qualifying exam for coding agents as robot-learning engineers. Each task provides an engineering specification, a containerized workspace, and task-specific tools and resource budgets, and the agent's solution is evaluated by its behavior in simulation, using visual observations alongside text across repeated cycles of observation, action, and experimentation.
Nine task families span four engineering categories: interactive control (kitchen manipulation, reusable harnesses, embodied reasoning), policy development (humanoid motion tracking, VLA policy training under a metered replay budget), perception and estimation (pose estimation under adversarial sensing, cluttered-bin clearing), and mechanical design (a mobile manipulator base, and co-designed teleoperation hardware and software).
Task-specific metrics are aggregated into an overall RLE Index with workflow-specific capability profiles, and the paper adds case studies of agent behavior on representative tasks. The repository provides the task families, simulation environments, scoring harnesses, and a CLI for running evaluations with Harbor, and a leaderboard is published at rle-bench.github.io.
Alternatives
Compare side by side- DFM-Redesign
DFM-Redesign benchmarks intent-preserving CAD redesign for manufacturability, while RLE-Bench tests coding agents as robot-learning engineers.
- cadgenbench
CADGenBench scores AI-generated and edited 3D parts against reference geometry, while RLE-Bench tests coding agents as robot-learning engineers.
- DepthBenchCAD
DepthBenchCAD audits how deeply CAD generations are checked, while RLE-Bench tests coding agents as robot-learning engineers.
- PCEval
PCEval tests physical computing with circuits and code in simulation, while RLE-Bench tests coding agents as robot-learning engineers.
Sources
Not verified
- Pricing: none stated on the repo
- Real user voices (no real linked post read for this page)
- Related Error Index entries (no matching entry verified)
- The catalog's 48 agentic tasks figure (the README lists nine task families; no 48-task count was confirmed in the sources read)
- A named individual builder beyond the RLE-Bench organization
Last verified November 6, 2026. Listings are informational. The evidence score measures public, verifiable evidence and is not an endorsement or a compatibility claim.