TEST INFRASTRUCTURE FOR PHYSICAL AI

Test every change
before it reaches a robot.

Invariant is test and evaluation infrastructure for robots and other physical AI. Run your autonomy stack, perception model or manipulation policy through repeatable scenarios, score it against ground truth, and see exactly what changed. Built for teams shipping faster than they can validate.

01

While you develop

Run any scenario from your laptop and see the result in minutes.

02

On every commit

Gate merges on the metrics your team cares about.

03

Nightly, at scale

Run your full suite of scenarios and trials in parallel.

04

Before release

Sign off on a build with evidence, not a gut feeling.

AUTONOMY · LOCALIZATION

Every autonomy build,
tested before it ships.

Drive the same routes in simulation or replay recorded logs, from your laptop or in CI. Invariant scores pose error, clearance and goals reached against ground truth, and compares them with your last good build.

$ invariant eval --stack nav_1.4.3 --compare 1.4.2
planned routetrue pathlocalization error
5 m
SIM · warehouse_nav_0400:00 / 01:47
metric1.4.31.4.2
localization RMSE–0.04 m
min clearance–0.43 m
goals reached0 of 99 of 9
thresholdRMSE ≤ 0.10 m
running
Illustrative example
PERCEPTION · DETECTION

Catch regressions
before the field does.

Run your detector on the same labeled scenes, simulated or recorded, while you train and on every commit. Every detection is scored against ground truth, and a scene that passes on main but fails on your branch blocks the merge.

$ invariant eval --model detector_v7 --compare main
front_camera · aisle_day_01detector_v7
person 0.94 · IoU 0.87 pallet 0.89 · IoU 0.82 crate 0.91 · IoU 0.78
┄ ground truth  ━ detection  ┄ missedIllustrative example
scenes0/50
modelv7 vs main (v6)
metric · 3 objects/scenev7main
recall–1.00
low-light recall–1.00
mean IoU–0.80
false positives–0
passing
scoring against ground truth…
MANIPULATION · POLICY EVAL

Grade every policy
before it grabs anything.

Evaluate every checkpoint on the same pick-and-place tasks, hundreds of trials in simulation, scored per object against the last one. A better average can't hide a new failure.

$ invariant eval --policy pick_v12 --compare main --trials 500
pick_and_place · tabletop_simlast 25 trials per object
objectv12vs main
grasp + place  dropped / missedIllustrative example
trials · 100 per object0/500
policyv12 vs main (v11)
metricv12main
overall success–87.4%
mug success–88%
collisions–0
running
running trials…
HOW IT WORKS

One harness, from first commit to release.

01

Connect your stack

Point Invariant at your autonomy stack, model or policy. Works with ROS 2 and rosbag2 (MCAP) recordings.

02

Run scenarios

Run the same scenarios, simulated or replayed from logs, with ground truth for pose, objects, paths and task success.

03

Compare and decide

See what got better or worse against any baseline, and stop regressions before they reach a robot.

Looking for a few teams to pilot with.

We're building Invariant with a small group of design partners across autonomy, perception and manipulation. If testing can't keep up with how fast you ship, let's talk.

or email founders@invariant.run