LeCun on Turing Award Acceptance and the State of Deep Learning

LeCun's Turing Award lecture, delivered jointly with Yoshua Bengio and Geoffrey Hinton in 2019, provides an intellectual retrospective on three decades of deep learning research and a forward-looking perspective on what remains unsolved. LeCun's portion focuses on three things: (1) the history of CNNs and why they were initially dismissed by the AI mainstream despite working in practice; (2) the critical importance of self-supervised learning for the next phase of AI development; (3) what he calls 'the dark matter of intelligence' — the implicit, non-linguistic knowledge that humans have from grounded experience in the world, which text-only models cannot acquire. The lecture was delivered before ChatGPT, before GPT-4, before the LLM moment — which makes its predictions remarkably prescient. LeCun correctly anticipated that self-supervised learning would become the dominant training paradigm. He correctly anticipated that text-only models would have a knowledge gap related to physical world understanding. He also anticipated the importance of world models, though he didn't yet have the JEPA architecture to propose as the solution. The lecture is the best single document for understanding LeCun's intellectual commitments before the LLM era, and for evaluating which of his predictions have held up in the years since.