Waymax Safety Monitor · AI Builder Portfolio

Live capture of the safety monitor running the intersection_conflict scenario. The map view shows the ego vehicle and surrounding agents; the right panel lists trigger events as they fire.

Direct capture of the pygame dashboard, headless. 46 frames at 5 fps (stride 2 over the 10 Hz sim). Every element on screen (map, agents, playback bar, trigger panel, fleet panel) is drawn by the shipping app, not a mock.

Problem

Operators can't watch every screen.

Every commercial robotaxi service today keeps humans on standby to take over when the vehicle asks for help. As the ratio of vehicles per operator grows past a handful, no operator can keep eyes on every tile. The bottleneck isn't the driving. It's spotting which of the twenty tiles on the monitor needs a human right now, and why.

Existing tools show video and telemetry. They don't rank the fleet by risk, they don't explain the reason for the risk in one glance, and they don't help the operator rewind to understand what just happened.

Solution

A ranked fleet view with a reason for every alert.

The dashboard runs a trigger engine over the live scenario and emits severity-tagged events: AMBER for "watch this" and RED for "take over now." Each event names the specific condition (time-to-collision, off-road excursion, hard brake, pedestrian too close, and four others). A scrubbable playback bar lets the operator rewind and audit what led to the event.

The trigger engine is deterministic and re-armed on a clear window, so an alert doesn't spam-fire while a condition persists. Thresholds ship in three policy presets (conservative, default, permissive) so a fleet can dial its false-positive budget.

Success criteria

When we'd call this "working."

Every collision in a scenario is preceded by at least one RED event, with lead time long enough to hand over.
False positives are low enough that operators don't tune out.
The reason string on every event is specific enough that an operator knows what to look at without opening the log.
The same scenario re-run produces the same event stream, byte for byte.
The system runs headless (for regression tests) and interactively (for live operations) from the same code path.

Metrics, KPIs, evals

Numbers from the runs above.

KPI	Value	What it means
RED-event recall	3 / 3 scenarios	Every demo scenario that ends in a conflict fires at least one RED event before the conflict. No missed calls in the demo set.
RED events per scenario	8 / 9 / 12	intersection_conflict, pedestrian_crossing, hard_brake. This is the operator's workload budget. Higher is worse, not better.
AMBER-to-RED ratio	~2x–5x	Warnings should outnumber take-over calls. Ratio too low = the trigger is jumpy. Ratio too high = alarm fatigue.
Trigger coverage	7 kinds	Time-to-collision, bounding-box overlap, off-road, wrong-way, lane compliance, hard brake, VRU proximity. Composite risk blends them.
Reproducibility	deterministic	Fixed seed per scenario. Every event is reproducible from the trajectory log alone.
Headless capture	SDL dummy	Same code path renders the pygame window and dumps PNG frames for CI. The video above is one such capture.

What we haven't shipped yet.

Live WOMD scenarios end-to-end. The loader is wired; it needs a JAX/TF install to run against real Waymo tfrecords.
Multi-vehicle fleet panel with more than one active tile.
Operator ergonomics testing. The KPIs above are engineering KPIs, not user-research KPIs.

Other captures

pedestrian_crossing

9 RED events. Dominant trigger: TTC on jaywalking pedestrians. Overlap secondary.

hard_brake

12 RED events. Dominant trigger: TTC from the front, hard-brake fires on lead vehicle at t=3.1s.

Full event logs: intersection_conflict · pedestrian_crossing · hard_brake