An online anti-bunching copilot for bus drivers
Many people in Salvador, Brazil depend on public transport, and most of them travel by bus. The city also has heavy road traffic and is one of the more congested in the world. Large transport projects are hard to deliver here, because responsibility is split across many public and private bodies and the bus system is under serious financial strain. These conditions favour interventions that work with the buses and data that already exist, cost little, and do not rely on agencies coordinating with each other.
Six words you need before you read the dashboard
The picture below contrasts the service we want with the failure we are fighting. Scroll, and we will name each part.
Headway
The time gap between successive buses arriving at the same stop. Even headways — here, 15 min in the top row — are the entire goal of high-frequency service.
Bus bunching
The unstable state in which buses that should be evenly spaced clump into a platoon, leaving a large gap behind them — the bottom row, where four buses run one minute apart.
Dwell time
The time a bus spends stopped to board and alight passengers. Heavy boarding lengthens dwell — and dwell is a primary driver of bunching: a late bus collects the passengers an on-time bus would have carried, which slows it further.
Forward headway
The gap from a given bus to the bus ahead of it on the same route. This is the core signal the model predicts. In the bottom row it has collapsed to a minute inside the platoon while opening to 57 min behind the leader.
Driver nudge
The system's advisory output to one driver: “ease off / let the gap open” or “hold briefly”. It is advisory only — the driver stays in control — and it never asks the driver to speed up.
Bunching-risk classes
For the next window, the model predicts one of three labels. This traffic-light grammar runs through the entire page:
ok warning bunching
How one bus drifts off headway, component by component
Before the dashboard makes sense, watch the three numbers the model actually computes — built up one layer at a time.
Forward headway
Start with the raw gap: the minutes since the previous bus passed this same stop. As our trailing bus closes on the one ahead, this gap shrinks.
The local “normal”
A raw gap means nothing on its own — peak gaps are short, late-evening gaps are long. So we compare it to the local-normal headway: the rolling median of the last eight gaps at this stop.
The headway ratio
Divide one by the other: ratio = realised gap ÷ normal. A ratio of 1.0 is on-headway; below 1.0 the bus is running closer than usual — closing on the bus ahead.
Two thresholds
The ratio is graded with two cut-points the whole system shares:
ratio ≥ 0.60 0.40 – 0.60 < 0.40
Cross 0.60 and the light turns amber; cross 0.40 and it turns red. That is exactly what the dashboard's HUD shows next.
Put it together: drive the copilot yourself
Three buses on a 60-minute loop. The HUD is the signal you just learned, live. Follow the steps — or grab the controls and explore.
Ease off only — never speed up.
Press play, or scrub
Hit ▶ to animate the 60-minute window, or drag the slider to any moment. Right now the three buses are evenly spaced and every reading is green.
Watch Bus 2 drift
As time passes, Bus 2's forward headway shrinks — it is catching up to Bus 1 while a gap opens behind Bus 3. The ratio falls, and the risk light turns amber, then red.
Apply the nudge
When the Ease off / Hold briefly button appears, click it. Bus 2 eases off, the gap reopens, and the light returns to green.
The action space is deliberately safe: ease off or briefly hold — never speed up.
Compare the scenarios
Toggle Corrected scenario to replay the same timeline with the auto-nudge firing at the warning threshold — and see even spacing preserved throughout, before bunching ever forms.
The system is advisory only: it informs the driver, who remains in control. The action space is deliberately safe — ease off or briefly hold, never speed up. — Methodology, §3.3 “Driver nudging as online classification”
First, is bunching even real here? And what does it cost?
Measured directly from reconstructed forward headways on the eight busiest routes — no machine learning yet. Every statistic is computed within hourly windows, so we never mistake the normal daily demand cycle for bunching.
Headway irregularity (CV)
The standard measure of spacing irregularity is the coefficient of variation, CV = std(h) / mean(h). CV = 0 is perfectly even service; bunching inflates it.
A median CV of 0.73
Across the eight routes the median within-window CV is 0.73 (mean 0.80). The irregularity is worse in the peak — where most passengers travel — rising from 0.79 off-peak to 0.86 in the peak.
How close the buses run
For every arrival we compute the same headway ratio the model uses, and plot the whole distribution. The dashed lines are the warning and bunching thresholds.
Nearly a fifth already bunched
By the model's own thresholds, 19.4% of arrivals are already bunched (ratio < 0.40), and a further 9.8% are in warning (ratio < 0.60). Almost three arrivals in ten are running closer than they should.
Avoidable passenger waiting
Irregular spacing converts straight into wasted time. Renewal theory splits a passenger's mean wait into an even-service floor plus an avoidable term driven purely by irregularity.
Half a million minutes a day
On these eight routes, irregular spacing imposes on the order of 5.6 × 10⁵ avoidable passenger-waiting-minutes per day — about 9,400 passenger-hours daily, or roughly 17 avoidable minutes per waiting passenger.
Severity over the day
Following Enayatollahi et al. (2019), severity is max(0, 1 − headway ratio): zero on schedule, rising toward 1 as a bus closes up on the one ahead.
It peaks with the commute
Averaged by hour, severity peaks in the morning and evening commute. Bunching is worst precisely when the most people are waiting — the two shaded humps above.
Seeing bunching directly
The per-bus string diagram shows the same phenomenon in the raw trajectories. Each line follows one vehicle; converging or crossing lines are bunching events.
Only OD data keeps this signal
This structure is visible only because the Origin–Destination records keep each bus individually, with exact arrival times. Aggregating into 5- or 30-minute per-interval counts destroys it — you can no longer tell a 30-second gap from a four-minute one.
From a raw stream to an online predictor — and the metric that decides
Before any model can learn, the data itself has to be understood: it arrives as a stream, it comes from a specific dataset, and it has to be reshaped into features. We walk that path — from raw arrivals to an online predictor — and then to the metric that actually decides which model to trust.
An unbounded, shifting stream
Each bus arrival is one event in an unbounded, non-stationary stream. The arrivals never stop, and the patterns that govern them shift with time of day, demand, traffic and weather. A model trained once on a fixed sample would slowly fall out of step with the world it is meant to watch.
Test, then train — in order
This setting calls for prequential evaluation: for each arriving bus the model first predicts, the bus then arrives, and the model finally learns from the realised outcome before moving to the next. Processed in strict chronological order, this reproduces the score the model would have earned running live.
A random split would leak the future
Shuffling the arrivals into a random train/test split would let the model learn from events that, in real time, have not happened yet. Because bunching is defined by what a bus does next, that leakage would inflate the score. Keeping the stream ordered is what makes the evaluation honest.
Salvador's raw arrival records
The data is the Salvador Urban Network Transportation (SUNT) dataset — public Origin–Destination records in which every row is one bus arriving at one stop. Crucially, each row carries an individual vehicle id, a trip id, the route and direction, an exact arrival time, the stop position, and the onboard load.
Eight routes, three days, 109,029 arrivals
From this release we take the eight busiest routes over 1–3 March 2024. After cleaning, this yields 109,029 arrivals, ordered by time. It is enough to expose the daily demand cycle and the drift that comes with it, while staying small enough to reason about.
The aggregated tables lose the identities
The assignment also supplies SUNT pre-aggregated to 5- and 30-minute intervals. Those tables are keyed by (stop, interval) and store only per-interval counts — boardings, loading, vehicle counts. They carry no vehicle id and no exact arrival time.
Aggregation destroys the signal
If two buses pass a stop in the same window, the aggregate cannot tell whether they were 30 seconds apart (bunched) or four minutes apart (normal). Aggregation therefore destroys the very signal bunching is defined on, and no post-processing can recover it. That is why the pipeline works from the raw records.
Rebuild the headway from arrivals
The pipeline first drops trips with out-of-order timestamps (about 23% of the raw trips), then reconstructs, for each arrival, the forward headway — the gap to the previous distinct vehicle at that stop — and the local-normal headway, the rolling median of the last eight gaps there.
Six features, all causal
From these it derives six features that use only information available at or before the current arrival: headway_min, headway_ratio, headway_trend, since_prev_stop_min, pt_sequence, and is_peak. Restricting to the present prevents the future from leaking into the input.
A future-looking, three-class target
The label looks three stops ahead on the same trip. If the minimum future headway drops below 0.40 of the local normal it is bunching; below 0.60, warning; otherwise ok. The classes are imbalanced — roughly 70% ok, 10% warning, 20% bunching.
First, guess the arriving bus
With only its causal features in hand, the model predicts a risk class for the bus about to arrive — ok, warning, or bunching. This is the prediction it would issue live, before the outcome is known.
The bus arrives; learn from it
The arrival then reveals the true label, and the model updates on that single instance before touching the next one. Learning one arrival at a time is what lets the streaming learners track drift instead of freezing on a stale snapshot of the network.
Repeat, in order, 109,029 times
Predict, reveal, learn, advance — repeated over every arrival in chronological order. The running score this produces is exactly the prequential accuracy charted next, and the basis for the model bake-off that follows.
Learn online, adapt to drift
A model trained once would grow stale. These learners update with every new arrival, and ADWIN monitors the stream for distribution change, letting the trees self-replace when the world shifts.
It recovers after every drift
Accuracy climbs fast and holds near 0.95. The grey lines are ADWIN drift points; the brief dips around them — and the prompt recovery — are the adaptive learners doing their job.
Six learners, one stream
On 109,029 instances, an Adaptive Random Forest takes the top bunching-class F1 and balanced accuracy — but accuracy alone, you'll see, is not the point.
| Model | F1 | Bal. acc. | κ |
|---|---|---|---|
| ARF | 0.916 | 0.915 | 0.892 |
| static threshold | 0.909 | 0.904 | 0.883 |
| HAT | 0.907 | 0.899 | 0.879 |
| Hoeffding Tree | 0.886 | 0.791 | 0.792 |
| EFDT | 0.877 | 0.789 | 0.785 |
| KNN | 0.761 | 0.728 | 0.676 |
| Naive Bayes | 0.000 | 0.333 | 0.000 |
Skill alone is not enough
Notice the static threshold was competitive on F1. On a conventional accuracy metric the model looks only marginally better. So what actually separates them?
≈ 28 minutes of warning
ARF's nudges fire a median of ≈ 28 minutes before the observed bunching — early enough for a driver to ease off. The static threshold and persistence baselines have no meaningful lead: they flag bunching only as it happens.
A warning is only useful if it's early
For an intervention that depends on a driver having time to act, a predictor that fires early is worth more than one that scores a fraction of an F1 point higher but fires late. Lead time, not F1, is what decides whether a model can support a nudge.
If drivers act on the warning, does service improve?
The true effect can't be observed — SUNT has no period when the copilot was deployed. So we replay the recorded arrivals and, whenever ARF fires a nudge on a bus genuinely closing on the one ahead, ease that driver off by a few seconds (capped at 1.5 min per trip), then recompute downstream headways.
Before the numbers — exactly what the simulation does
The result rests on a specific, mechanical procedure: replay each recorded arrival, decide where a driver would have been nudged, add a small bounded ease-off, and recompute the spacing. We walk that procedure — and follow one bus through it — before reading the outcome.
There is nothing to measure directly
SUNT has no period when the copilot was deployed, so the effect of the nudge cannot be observed. Instead we take the recorded arrivals as they happened and ask, arrival by arrival, what would have changed had drivers acted on the warnings.
Read the predictions back, don't re-run
The model is not invoked again. The simulation reads the predictions the streaming run already produced — one pred_ARF per arrival, saved alongside the recorded headway and the local-normal reference — and replays the arrivals in trip order. This keeps the counterfactual honest: it uses exactly the live-earned predictions, not hindsight-tuned ones.
The model predicts a warning
The first requirement is a live prediction of at least warning — that is, pred_ARF ≥ 1 (warning or bunching). A prediction of ok never triggers anything.
…and the bus is genuinely closing
The prediction alone is not enough. The bus must also be actually closing on the one ahead: headway ratio < 0.60, using the same ratio the whole system speaks. This guard stops the ease-off from firing on a bus that is merely flagged but still well-spaced.
Both, together, at that stop
Only when both hold — warned and closing — is that arrival marked to act. Everywhere else the bus is left exactly as recorded, so the intervention is deliberately conservative.
Add a few seconds of dwell
When a stop acts, the driver eases off by a small, fixed amount — 8 seconds of extra dwell in the default setting. Nothing dramatic; just enough to let the gap ahead begin to open.
8, 16, 24 … up to a 90-second ceiling
If later stops on the same trip also act, the delay accumulates — 8 s, 16 s, 24 s, and so on. To keep the intervention realistic it is capped at 90 seconds (1.5 minutes) per trip: once a trip has been eased off by a minute and a half in total, no further delay is added.
It only ever slows a bus
The delay is always non-negative: the nudge can ease a bus off, never speed it up. This matters for realism — a driver can dwell a little longer, but cannot be asked to make up time by driving faster.
A bus closing on the one ahead
Follow one trip along its stops. On the recorded baseline the trailing bus is closing — the gap to the bus ahead is shrinking stop by stop, and the ratio is sliding toward the warning band. (The diagram is idealised for clarity; the real trip has more stops and smaller shifts.)
The nudge fires — arrivals shift forward
At the first stop where the bus is warned and closing, the ease-off begins. The added dwell shifts this bus's arrival at that stop and every later stop forward in time — the accumulated delay travels with the bus for the rest of the trip.
The gap re-opens, headways recomputed
Because the bus now arrives a little later at each downstream stop, the gap to the bus ahead re-opens. Forward headways are then recomputed on the shifted arrival times, and it is these recomputed spacings — across every trip — that the results measure.
A stable yardstick
The local-normal headway — the "normal" each ratio is measured against — is not recomputed after the ease-off. Holding it fixed means the before and after are judged against the same reference, so the comparison reflects the intervention rather than a moving baseline.
Is the benefit real, and is it fragile?
Two guards frame the result. A true-positive-only run eases off only where the model was correct, testing whether the benefit comes from genuine prediction rather than hindsight. A sensitivity sweep varies the ease-off from 4 to 16 seconds, testing whether the result balances on a knife's edge. The next figures put numbers to both.
About one event in eight, gone
The share of arrivals that are bunched falls from 19.4% to 17.2%, and the raw count of bunching events from 21,123 to 18,463 (−13%) — because a closing bus eased off and let the gap reopen.
≈ 10% less avoidable waiting
Avoidable wait drops from 201,978 to 182,196 passenger-minutes (−9.8%), and mean headway CV from 0.796 to 0.768.
It's the prediction, not hindsight
Gating the ease-off to only the model's correct (true-positive) nudges gives virtually the same reduction (−12.7% bunching / −9.8% wait) — so the benefit is driven by real-time prediction, not by knowing the answer in advance.
21,123 → 18,463
201,978 → 182,196 pax-min
(% of arrivals)
(−3.5%)
Not balanced on a knife's edge
Does the result hinge on one lucky choice of ease-off size? We sweep the per-stop delay from 4 to 16 seconds.
Stable across the range
The bunching reduction stays in a narrow 12–13% band and the excess-wait reduction near 10% throughout. The estimate doesn't depend on a single tuned value.
The benefit is variance-driven, not a faster average
Easing a closing bus off slightly delays it, so the median wait barely moves (≈ 10.7 min before and after) and ride time is untouched. What falls is the avoidable, variance-driven portion of the wait. A flat median alongside a falling avoidable wait is exactly what the renewal model predicts — internal evidence the simulation is behaving correctly, not against it.
The nudge evens spacing, not driving speed. We claim no crowding benefit and no congestion benefit — in SUNT, occupancy barely tracks the headway (r ≈ 0.09) and boardings do not rise with the gap ahead (r ≈ 0.00), so there is nothing measurable there to improve. — Discussion, “What we deliberately do not claim”
A copilot, not a control room
A decentralised, advisory, zero-infrastructure copilot that predicts bunching early enough for one driver to act — reconstructing per-bus forward headways from SUNT OD data, learning online under prequential evaluation, and warning with actionable lead time. An Adaptive Random Forest gives the best imbalance-aware skill; a clearly-labelled counterfactual estimates ≈13% fewer bunching events and ≈10% less avoidable waiting.
Scroll back up and explore the simulation freely: scrub the 60-minute window, fire the nudge, toggle the corrected scenario, switch the theme. The whole page speaks the same traffic-light grammar — ok, warning, bunching.
↑ Back to the simulationThe experiment — preprocessing, the streaming models, and the counterfactual — runs entirely in Python; the streaming learning itself is done in CapyMOA (a Python interface to the JVM-based MOA engine). The report toolchain is excluded here.
A personal note
It was a great opportunity to study Machine Learning for Data Streams at the University of Waikato, one of the pioneering institutions in the field and a global centre for stream-learning research.
Waikato researchers helped develop MOA (Massive Online Analysis), an influential open-source framework introduced in 2010 for building and evaluating machine-learning models on continuously arriving data. That work continues through newer tools such as the Python-based CapyMOA library.
Learning directly from researchers behind methods we were studying, including ADWIN, Hoeffding Adaptive Tree, Leveraging Bagging, and Adaptive Random Forest, was both challenging and rewarding. These were not simply algorithms presented in a textbook. They were ideas developed and refined by researchers connected to the university and the course itself.
What I valued most, however, was the opportunity to apply machine learning for data streams to a real-world dataset such as SUNT.
For our project, we built a model that continuously learned from public-transport data to predict when buses were at risk of bunching together. We then explored how those predictions could support an early-warning system that gives drivers enough time to respond.
The course was difficult, but it demonstrated what makes this area of machine learning so exciting. The goal is not simply to train a model once and improve a metric. It is to build systems that can keep learning as the world changes and potentially help solve problems that affect people every day.
“Very interesting topic … this group scored the best overall.” — Marker's feedback, Assessment 3
Evidence & source traceability
This explainer paraphrases the report for a general reader. Each major section carries a Report § chip that opens the matching section of the paper (PDF). Claims fall into four kinds, kept visually distinct:
- Reported finding Reported finding. A number or result the authors state directly — e.g. the ≈28-minute median lead time or the ARF bunching-class F1 of 0.916.
- Simplified explanation Simplified explanation. Our paraphrase for the reader — not the paper's exact wording — such as the traffic-light framing of the risk classes.
- Interpretation Interpretation. A practical implication we draw beyond the paper's stated claims, offered to aid understanding rather than reported as a result.
- Limitation Limitation. A boundary on the conclusion. Author-stated limitations live in the “Limitations” paragraph (p.21) — most consequentially, the counterfactual is a simulation under stated assumptions, not a measured field outcome.