How do self-driving cars see in the dark?

Lidar brings its own light: each beam is a laser pulse, timed on its way back, so it measures distance the same by day or night. Waymo says its lidar sees objects up to 300 m away in all lighting conditions. Cameras add what lidar can't read, like a traffic light's color, and radar measures speed through fog and rain.

What happens when something is hidden behind a parked car?

No beam reaches it, so the car can't know it's there. Stacks handle that in two ways: they remember an object that slips behind another for a moment (this model keeps a lost track for 1 s), and they slow down near places someone could step out from, as this model does beside the van.

How far ahead does a self-driving car look?

In time, a few seconds, re-planned many times a second. This model plans 3 s ahead and decides again every half second. Shorten the horizon and trouble shows up late: looking 1 s ahead at the left turn, the car commits before it sees the oncoming car's path cross its own, and hits it.

Do self-driving cars follow traffic rules?

In a planner like this one, a rule is a cost. Crossing the stop line on red costs 50,000 here, more than any amount of progress earns back, so at the yellow light the car eases off from 47 km/h, stops at the line and waits for green.

Live model · Self-driving cars

How Self-Driving Cars Actually Work

A self-driving car runs one loop over and over. Its lidar measures distances all around it; points that aren't road or buildings become boxed objects; each object's motion is predicted a few seconds ahead; and the car tries several plans against those predictions, scores each, and drives the cheapest until it looks again.

One intersection, half a second at a time

One lidar sweep: 800 beams fan out all the way round, each stopping at the first thing it hits within 50 m. 732 points come back, 711 of them on buildings the map already has. 3 things in the street got no point at all: the dashed boxes.

StopPerceive, predict, plan

WhereLidar range (m)Lidar noise (cm)Predict ahead (s)CautionPrediction

Time0.0 sSpeed40 km/hLightgreenObjects2

What the car sees, and what it plans

Lidar pointsThere, not seenGrid: 1 m

Five plans, scored

Plan	Cost	Why
Speed up	·	·
Hold speed	·	·
Ease off	·	·
Brake	·	·
Hard brake	·	·

Speed, km/h

speed limit
speed

CH1 Detected, still
CH2 Detected, moving
CH4 Speed

How to read the stage

The car (cyan) drives north at up to 45 km/h. Every half second its 800 lidar beams sweep the street: yellow boxes are still objects it detected, magenta ones are moving, with dashed paths and discs where they may go. Dashed outlines are things no beam reached. At 2.5 s a walker steps out from behind a parked van. The car brakes hard twice and comes no closer than 1.1 m.

Seeing: one sweep

A lidar fires laser pulses and times each reflection's return, so every beam gives one distance. Waymo's reaches past 300 m; the 32-beam unit on the nuScenes research car spins 20 times a second out to 70 m. Here, from 20 m back, 800 beams return 730 points, 590 on buildings the map knows. The walker gets none: the van is in the way. Cut to 200 beams and the person on the corner falls between two.

One lidar sweep, beam by beam

The car stands with its front 20 m before the stop line, 1 s into the scene. Its lidar sends 800 beams around it, about one every 0.45°, out to 50 m.

StopLidar

WhereCar before the line (m)Seconds inLidar range (m)Lidar noise (cm)Beams

Object	Points	Size	Score
(none)

CH1 Detected

Deciding: five plans, scored

Planning here is a search: five speed plans along the route, each driven 3 s forward in eighth-second checks against every prediction. Overlapping a prediction costs 25,000 a check; passing within 1.5 m costs more the closer; braking costs its square; progress earns points back. At 4.0 s the cyclist is crossing the car's path. Holding speed would hit it, so braking at 4 m/s² wins, at −101.

One decision, taken apart

4.0 s, 41 km/h: the moment it decides. It tracks 4 objects, 1 of them moving, and predicts each one on at its measured velocity, its disc of doubt growing with every half second, 3 s ahead.

StopPlan

WherePredict ahead (s)CautionPrediction

Time4.0 sSpeed41 km/hHorizon3 s

The car at 4.0 s, deciding

Lidar pointsWhere it may beGrid: 1 m

What it tracks

Object	Away	Speed
#3	22.7 m	18 km/h
#2	5.8 m	still
#4	16.7 m	still

Each plan's cost (lowest wins)

Plan	Crash	Caution	Rules	Comfort	Progress	Total
(none)

CH2 Moving, predicted
CH3 Plan being scored

Driving for what it can't see

Waymo says its Driver doesn't outdrive what it can see. The model's version: driving fast beside something big and still, like a van that could hide someone, costs the caution weight times the speed squared. At caution 4 the car eases off by the van and keeps 1.1 m from the walker; at 0 it comes within 0.5 m. Prediction matters as much: switch it off and the car hits the cyclist.

What the model leaves out

Real stacks fuse lidar with cameras and radar, recognize objects with trained networks, track them with filters that weigh each noisy measurement, and predict several possible futures for every road user. This model sees only lidar, boxes points by touching squares, scores confidence by point count, predicts one straight path, and picks one of five decelerations along a fixed route. Footprints are circles, and everyone else follows a script.

Questions

How do self-driving cars see in the dark?: Lidar brings its own light: each beam is a laser pulse, timed on its way back, so it measures distance the same by day or night. Waymo says its lidar sees objects up to 300 m away in all lighting conditions. Cameras add what lidar can't read, like a traffic light's color, and radar measures speed through fog and rain.
What happens when something is hidden behind a parked car?: No beam reaches it, so the car can't know it's there. Stacks handle that in two ways: they remember an object that slips behind another for a moment (this model keeps a lost track for 1 s), and they slow down near places someone could step out from, as this model does beside the van.
How far ahead does a self-driving car look?: In time, a few seconds, re-planned many times a second. This model plans 3 s ahead and decides again every half second. Shorten the horizon and trouble shows up late: looking 1 s ahead at the left turn, the car commits before it sees the oncoming car's path cross its own, and hits it.
Do self-driving cars follow traffic rules?: In a planner like this one, a rule is a cost. Crossing the stop line on red costs 50,000 here, more than any amount of progress earns back, so at the yellow light the car eases off from 47 km/h, stops at the line and waits for green.