Emergent tactics in Formula One race strategy via reinforcement learning

In brief

• Formula One pitstop strategy is inherently a multi-agent game: an optimal decision depends on anticipating the reactions of rival pit walls.
• Existing industry methods rely on linear programming or isolated Monte Carlo rollouts that miss dynamic opponent interactions.
• This work introduces a modular architecture connecting Mercedes' proprietary Monte Carlo simulator with reinforcement learning via Unified Race State (URState) and Unified Race Strategy (URStrategy) abstractions.

This retrospective explainer is dated to the June 2026 publication in the journal Machine Learning. It discusses Race Strategy Reinforcement Learning: Optimising Pitstop Strategy with Emergent Tactics in Formula One by Devin Thomas, Junqi Jiang, Avinash Kori, Aaron Russo, Steffen Winkler, Stuart Sale, Joseph McMillan, Francesco Belardinelli and Antonio Rago.

In elite motorsport, there is a legendary tactical maneuver known as the undercut.

Suppose Driver A is trailing Driver B by 1.5 seconds. The aerodynamic wake (“dirty air”) behind Driver B overheats Driver A’s front tyres, making an on-track overtaking move nearly impossible. Driver A pits one lap earlier than scheduled for fresh tyres. Returning to the circuit in clean air, Driver A unleashes an blistering out-lap on fresh rubber. When Driver B pits on the following lap, Driver A blazes past down the main straight before Driver B’s cold tyres can reach operating temperature.

Executing an undercut requires split-second timing. If Driver A returns into a pack of slower midfielders, the tactic backfires catastrophically.

While earlier work explored isolated RL agents on synthetic tracks, real-world Grand Prix strategy requires handling heterogeneous data feeds—from live telemetry to commercial racing simulators (F1 23) to high-fidelity team tools. This journal article presents the full end-to-end framework developed in collaboration with industrial motorsport engineers, uncovering emergent multi-agent tactics.

Bridging telemetry and RL: URState and URStrategy

Real-world racing environments present extreme engineering heterogeneity:

• Live track GPS telemetry provides millimeter-accurate car positions at 50 Hz.
• Driver-in-the-loop (DiL) simulators test human physiological reactions.
• Proprietary tools (such as Mercedes’ Monte Carlo Race Simulator) model fuel burn, thermal degradation, and safety car likelihoods.

To enable reinforcement learning agents to operate seamlessly across these disparate sources, the authors created two standardized interface abstractions: the Unified Race State (URState) and the Unified Race Strategy (URStrategy).

Figure from the paper. System architecture overview. Live race data, Driver-in-the-Loop simulator data, and F1 game data are ingested through dedicated state translators. The Race Simulator Translator connects Mercedes' Monte Carlo Race Simulator with the central Race Strategy Prediction Model via Unified Race State (URState) and Unified Race Strategy (URStrategy) interfaces, outputting actionable advice to the team strategist and driver. Source: Devin Thomas et al.. Extracted from the source manuscript; click to enlarge.

The URState compresses raw multi-modal telemetry into a canonical feature representation:

• Gap to leader and car ahead/behind in seconds and distance.
• Tyre compound age, thermal history, and compound degradation phase.
• Track status (Green, Yellow, Full Course Yellow, Virtual Safety Car, Safety Car).
• Projected pit exit window relative to the midfield pack.

The URStrategy action space encodes discrete strategic directives:

• Target compound: ${\text{Soft}, \text{Medium}, \text{Hard}}$.
• Target pit window: ${l, l+1, l+2}$.
• Stint pacing mode: ${\text{Conserve}, \text{Push}, \text{Box Now}}$.

Multi-agent game dynamics and emergent tactics

Unlike standard single-agent RL where the environment is stationary, a Grand Prix is a non-stationary, imperfect-information Markov game:

\[\max_{\pi_i} \mathbb{E}\left[ \sum_{t=0}^T \gamma^t \, R_i(s_t, a_{i, t}, \mathbf{a}_{-i, t}) \right],\]

where $a_{i, t}$ is the ego team’s action and $\mathbf{a}_{-i, t}$ are the simultaneous actions of all 19 rival teams.

When trained against ensembles of heuristic and learned rival policies, RSRL developed sophisticated emergent behaviors that were never explicitly programmed into the reward function:

1. Defensive In-Lap Covering: If a car behind is within the 1.8-second undercut threshold and enters the pit entry lane, RSRL reacts within 200 milliseconds to box the ego car on the next lap if track position can be defended.
2. Virtual Safety Car (VSC) Opportunism: Under VSC conditions, cars travel at roughly 60% of race speed. Pitting under VSC costs only 11–13 seconds of track delta compared to 22 seconds at racing speed. RSRL dynamically delays scheduled pitstops by 3–5 laps if the probability of an imminent safety car exceeds a threshold derived from track crash history.
3. Tyre Phase Management: When running in dirty air behind a rival, the agent reduces its micro-pace by 0.2 seconds per lap to preserve tyre core temperatures, extending the stint until clean air opens up after the rival pits.

Trust and operational deployment

Deploying AI on an F1 pit wall requires instant interpretability. A strategy recommendation that takes three minutes to explain is useless when a driver is approaching the pit entry at 280 km/h.

By coupling the RL policy with contrastive rule distillation, the system translates policy logits into concise tactical rationales displayed on the strategist’s dashboard:

“Box Lap 24 for Medium: Covers Hamilton’s undercut; 2.4s buffer ahead of Alpine traffic upon exit.”

This human-in-the-loop validation demonstrates how reinforcement learning can transition from academic benchmark simulators to mission-critical industrial decision systems.

Paper and sources

• Devin Thomas, Junqi Jiang, Avinash Kori, Aaron Russo, Steffen Winkler, Stuart Sale, Joseph McMillan, Francesco Belardinelli and Antonio Rago. Race Strategy Reinforcement Learning: Optimising Pitstop Strategy with Emergent Tactics in Formula One. Machine Learning 115, 146 (2026).