Generative Flow Matching based Motion Planner for AV

A learned motion planner for an autonomous vehicle. The planner uses flow matching with a tranformed based arch to generate control sequences. It generates 16 possible 8 s trajectories, and a rule based scorer selects one of them. The model is trained on 325 hrs of nuPlan driving data from four cities. Evalueated using nuPlan devkit's own simulator and metrics on the Val14 benchmark.

Results

Results are from a model fine tuned using a DAgger framework. Scores Non-reactive- 90.15 and Reactive- 88.46 on Val14 dataset using nuPlan closed loop metrics.

Val14 benchmark

Val14 (Dauner et al., 2023) has 1,118 scenarios from 328 validation logs. The devkit's own simulation and metric engine give all values.

Model	Non-Reactive	Reactive
DAgger round 2, PDM style scorer	90.15	88.46
pose perturbation, PDM style scorer	89.01	88.19
log replay (expert)	93.53	80.32

metric(means)	pose perturbation, non-reactive	DAgger round 2, non-reactive	pose perturbation, reactive	DAgger round 2, reactive	expert, non-reactive
no at fault collision	0.956	0.955	0.950	0.943	0.988
drivable area	0.987	0.988	0.984	0.983	0.979
driving direction	0.994	0.995	0.997	0.997	0.991
making progress	0.995	0.996	0.995	0.995	1.000
time to collision	0.894	0.889	0.889	0.886	0.943
speed limit	0.994	0.993	0.994	0.992	0.965
progress on route	0.912	0.954	0.889	0.926	0.990
comfort	0.992	0.996	0.996	0.995	0.993

Open loop

Done on all held 14,569 validation scenes.

model	meanADE	straight	turn	minADE
constant velocity	4.37	3.60	9.14
imitation	1.67	1.54	2.45	0.55
turn weighted	1.61	1.52	2.15	0.51
pose perturbation	1.63	1.54	2.18	0.52
DAgger round 1	1.91	1.85	2.30	0.56
DAgger round 2	1.90	1.84	2.28	0.56

Closed loop

Done only on 22 validation scenarios, 15 sec each, scored with the official nuPlan metrics (non-reactive only). Refer to Val14 number for proper eval numbers.

model	median-length sample	PDM-style scorer
imitation	0.75	0.87
turn-weighted	0.78	0.79
pose perturbation	0.86	0.97
DAgger round 1	0.80	0.87
DAgger round 2	0.85	0.87

Notes

• Pose perturbation stage gave the largest closed loop gain of the training stages. The open loop error did not change.
• Using DAgger, the median speed error decreased from 1.74 m/s to 0.79 m/s. States more than 1 m to the side decreased from 17.7% to 13.4%. On Val14, the score increased by 1.1 points (non-reactive) and 0.3 points (reactive).
• Providing no ego pose history helps the model must use the scene instead of extrapolating from history.
• After adding turn based weight in the loss function, the open loop turn error decreased.
• After DAgger, in open loop meanADE increased.
• For DAgger, to get an oracle for the recovery we used the afam optimizer and tried to optimize the original contraol sequence to get close to the experts path towards the tail. Kind off dtnamic weight to close the lateral error and heading error. Places where this was not achieved were rejected from being used in training.
• During training instead of sending all 769 tokens, we see how may tokens are null and send tokens = multiple of 16 to bake training faster, N*16 is selected based on the max in that batch. About 70% of the scene tokens were padding. Dropping them before the transformer made it 1.9 times faster.
• Number of data going in is selected based on proper budgiting using p99 coverage for the scene.
• The first model predicted waypoints, and the plans were not drivable (sideways acceleration of 34 m/s², against 0.43 m/s² for the human driver). Predicting acceleration and steering and running them through a bicycle model fixed this. Every sample is now within nuPlan's comfort limits.
• There was lot of overshoot on turns, weighted turn loss help fix it a tittle bit.
• Running the model with no steering state provides gives same output as with steering provides, shows the scene is used to plan.
• 325hrs is too less. But the outputs have defenitely learned a lot. 325hrs = 1.1 million scenes.
• The logged acceleration and yaw rate signals do not reproduce the driven path (0.56 m error). Fitting the controls to the path through the bicycle model does (0.007 m).
• Hiding each ego state input 30% of the time worked better than 75% (some paper has used 75%).
• (Limitation) The model does not predict the future motion of other road users. The logs do provide the future of other agents and the model can we trained to predict that. Its should help more with avaoiding collisions
• (Limitation) Closed loop positions are stored as float32, which is only accurate to about 0.5 m at Boston map coordinates.

Approach

Input

The scene is a set of 769 tokens

token type	count
ego state (speed, acceleration, steering)	1
road users, 2 s history (vehicles, pedestrians, cyclists)	232
static objects	160
lanes, 30 points each, with traffic light state	224
map areas, 20 points each (crosswalks, walkways, car parks, stop lines, intersections)	152

Model

• Each scene element becomes one token. The ego's speed, acceleration and steering are embedded separately and pooled by attention. Each road user's 2 s history and each static object go through an MLP.
• Lanes and map areas go through a polyline encoder- an MLP on every point(x, y, sin, cos), then a max over the points. Lane attributes (route, traffic light, speed limit) are added after pooling, so a single flag is not lost in the max.
• There is no positional encoding over token order. Positions are input features in the ego's frame. A learned embedding per token type (ego, road user, static object, lane, map area) tells the encoder what each token is.
• A transformer encoder reads the scene tokens.
• A transformer decoder gives a velocity field over the controls.
• Conditional flow matching (Lipman et al., 2023) trains this field to move Gaussian noise to the expert's controls.
• Size: 4 encoder and 4 decoder layers, width 256, 8 attention heads, 9.3 M parameters.

Output

• Longitudinal acceleration and steering angle, at 10 Hz for 8 s (80 steps).
• A kinematic bicycle model integrates the controls. Thus each sample is a trajectory that a car can drive.
• 16 samples per planning call, 10 Euler steps(emperically found), a new plan every 0.1 s.

Training

There are three stages. Each stage starts from the model of the previous stage.

1. Imitation. The model learns from the logged expert. Turn scenes have 2.5 times the weight in the loss.
2. Pose perturbation. In half of the training scenes, the ego is moved sideways and turned. The target is a recovery path back to the expert path, fitted through the bicycle model.
3. DAgger two rounds. The model drives training scenarios in the simulator i.e closed loop. Each state that it reaches gets a recovery label to the expert path. Then the model is fine tuned on these labels and on logged data. A lot of filtering and check are done to make sure we dont train on bad examples.

Dataset

nuPlan v1.1, full training split. 6,141 logs, 1,172,563 scenes (325.7 h). Las Vegas, Boston, Pittsburgh and Singapore.

Sample selection

A PDM style scorer (from Dauner et al., 2023) checks each sample over the next 4 s. Other road users move at constant velocity. The scorer uses the definitions and weights of the nuPlan metrics.

• Gates based ont fault collision, leaves the drivable area, or enters an intersection on a red light.
• Score based on progress, time_to_collision, speed_limit, comfort. The scorer selects the sample with the highest score.
• Fallback, If no sample passes the gates, the scorer selects the sample with the latest first collision, then the shortest path. This gives the most time to brake.

Repo Structure

Use main.ipynb to run the pipeline. The pipeline is designed to work with google drive

main.ipynb       Run this: data -> checks -> training -> open loop -> closed loop -> DAgger -> Val14 -> results
config.py        Drive layout, nuPlan environment, extraction settings (they key the tensor cache)
download.py      nuPlan maps, train split index, log fetcher, Val14 logs
data.py          Scene tokens of one sample. The same code builds the planner's input in closed loop
prep.py          Parallel extraction into the Drive cache, training and validation sets
controls.py      Control targets through the bicycle model, control scale, ego-state input
perturb.py       Pose-perturbation recovery targets
loader.py        Streaming training loader
flow.py          Conditional flow matching- path, loss, Euler sampler
model.py         Scene encoder, velocity field decoder, checkpoint loading
loss.py          Flow matching loss with the position term and turn weights
train.py         Training loop, smoke test, the training runs and the model registry
planner.py       Trajectory sampling and the nuPlan planner
simulation.py    nuPlan closed loop simulation and metrics
scorer.py        PDM style sample selection
closed_loop.py   The 22 scenario development set: stored runs, parallel simulation, tables
metrics.py       Open loop metrics and tables
dagger.py        DAgger rollouts, relabelling and fine tuning
val14.py         Val14 benchmark
visualize.py     Training curves, scene plots, closed loop gifs
diagnostics.py   The analyses behind the notes below
outputs/         Closed loop gifs of the development set
Generative_Flow_Matching_Motion_Planner.ipynb   The old/original single notebook version of this project

References

• Flow Matching for Generative Modeling
• nuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles
• Parting with Misconceptions about Learning-based Vehicle Motion Planning
• Rethinking Imitation-based Planner for Autonomous Driving
• ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
• Diffusion-Based Planning for Autonomous Driving with Flexible Guidance
• PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving
• Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling