Overview
A single success rate hides why a robot failed. It might not have seen an obstacle, picked a bad plan, or planned well and driven it badly. Each of those needs a different fix, so the environment looks at each one separately.
[02] Project
VLA Robotics Evaluator
PythonEmbodied AIVLASimulation
A simulated test environment for vision-language-action robots. The robot sees the scene as a flat top-down view and drives around obstacles to reach a green goal, and the evaluator breaks down why it fails when it does.
- Role
- Embodied-AI evaluation
- Year
- 2026
- Task
- Reach the goal zone
- View
- Top-down, 2D
- Failures scored
- Perception, planning, control
- Observe
- Plan
- Act
- Evaluate
How it works
The robot's only input is a top-down image of the arena: free space, obstacles and a green goal zone. From that image it produces actions to drive toward the goal, and the environment tracks what it saw, what it planned and what it actually did.
In this world
The robot runs episodes on its own. Some are clean, and some have a failure injected, like a hidden obstacle or a look-alike goal. The panel above the arena shows what the robot sees and which stage failed.