AgentSim Demo: Scenario Mutations of Real Drives, Rendered by a Video World Model

Same drive, different danger. For each of three real drives, AgentSim replays the logged drive (base) and several variants it wrote itself: a lead car that brakes hard, a car that cuts in, a red-light runner, a pedestrian crossing. Every variant runs in closed loop, so other road users follow their recorded paths until the mutation reaches them and then react. A video world model renders each variant as a 12-second front-camera clip. All clips of a drive start from the same first frame, so they differ only where the mutation changes the traffic.

AgentSim is the agentic scenario-generation module that Yilin Wang initiated within the TeraSim platform at Inchor (SaferDrive AI): a multi-agent framework driven by LLMs and VLMs that mutates logged drives into safety-critical corner cases for autonomous-vehicle testing.

How to read this page. Each section is one drive. The first row is the logged drive; the rows below it are AgentSim's variants. Clips are muted and loop; they start playing when scrolled into view. Expand Frame strip under a drive to see all of its clips as still frames at 0, 3, 6, 9 and 11.8 s.

1. Signalized intersection: the ego pulls away from a queue

Waymo Open Motion Dataset v1 validation, scene d1988864008fc062. 5 variants.

Logged drive

The logged drive: the ego pulls away from a queue and crosses a signalized intersection.

Variant 1

The lead car brakes hard at 3 s; the ego stops behind it.

Variant 2

A car cuts in from the next lane and brakes; the ego brakes.

Variant 3

A red-light runner crosses in front of the ego; the ego brakes.

Variant 4

A pedestrian crosses the road ahead; the ego yields.

Variant 5

The lead car brakes hard, then creeps on; the ego follows.

Frame strip: all 6 clips at 0, 3, 6, 9 and 11.8 s

2. Busy signalized intersection: the ego accelerates out of stop-and-go traffic

Waymo Open Motion Dataset v1 validation, scene 5c14062cdca66b03. 4 variants.

Logged drive

The logged drive: the ego accelerates from 12 to 53 km/h through a busy signalized intersection.

Variant 1

The lead car brakes hard right away and stays stopped; the ego stops behind it.

Variant 2

A car cuts in from the next lane at 2 s and brakes hard; the ego brakes to a stop.

Variant 3

A red-light runner crosses in front of the ego at 4 s; the ego brakes to 9 km/h.

Variant 4

A pedestrian crosses at the crosswalk ahead; the ego stops for them.

Frame strip: all 5 clips at 0, 3, 6, 9 and 11.8 s

3. Wide signalized arterial: the ego accelerates to 58 km/h

Waymo Open Motion Dataset v1 validation, scene 8bdbbe8276d03a1f. 3 variants.

Logged drive

The logged drive: the ego accelerates from 30 to 58 km/h along a wide signalized arterial.

Variant 1

The lead car brakes hard at 2.5 s; the ego slows to 15 km/h behind it.

Variant 2

A red-light runner crosses the intersection ahead at 5 s; the ego brakes.

Variant 3

A pedestrian crosses the road ahead; the ego brakes to walking pace.

Frame strip: all 4 clips at 0, 3, 6, 9 and 11.8 s