Self-Supervised Learning: The Dark Matter of Intelligence

LeCun's foundational essay on self-supervised learning (SSL) — the technique that powers BERT, GPT, and nearly every large language model. The essay is remarkable because it was written by one of the architects of modern deep learning before the ChatGPT moment, and it correctly predicts that SSL will become the dominant training paradigm. The core insight: supervised learning requires expensive human labels and is sample-inefficient. Reinforcement learning is notoriously unstable and requires too many trials. Self-supervised learning — where the model predicts parts of its own input from other parts — is how humans and animals actually learn: by observing the world and building internal models. LeCun uses the 'dark matter' analogy because most intelligence is implicit (learned without explicit supervision) just as most of the universe's mass is invisible. The essay predicts that SSL applied to images, video, and multimodal data will be the next major advance — a prediction that proved correct with CLIP, Flamingo, Stable Diffusion, and Meta's own work on video SSL. What the essay doesn't predict: that SSL on text (next-token prediction) would advance so rapidly that it would create a public perception that LLMs are intelligent. LeCun's subsequent work clarifies why text SSL alone is insufficient: intelligence requires grounded world models, not just statistical patterns in text.