Human-level performance in 3D multiplayer games with population-based reinforcement learning

This 2019 Science paper describes capture-the-flag (CTF) agents that can beat professional human players at a fast-paced 3D first-person shooter — a category of problem requiring teamwork, spatial navigation, and reactive coordination that was well beyond AI capability at the time. The key technical innovation is population-based training: instead of training a single agent, train an entire population of agents that compete against each other with varying skills and strategies, then select and reproduce the most successful variants. This creates a form of natural selection that produces diverse, adaptive strategies. The CTF paper is significant in Hassabis's arc because it extends superhuman AI performance beyond games of perfect information (Go, chess) to imperfect information, 3D spatial environments, and multi-agent coordination. The agents learn human-like behaviors — communicating to coordinate attacks, setting ambushes, defending flag carriers — not through explicit programming but through reward maximization. This was a proof point that RL can produce complex multi-agent coordination without any pre-specified social norms or communication protocols.