Yann LeCun's Cake" Problem (Joint Embedding Predictive Architecture)

LeCun's 'cake' analogy reframes the entire AI research agenda: most of intelligence is learning world models (the cake), very little is learning behavior (the frosting), and reinforcement learning is just the cherry on top. This talk from 2022 lays out JEPA — Joint Embedding Predictive Architecture — as the alternative to generative models for learning world models. JEPA learns abstract representations of the world by predicting representations rather than predicting raw pixels or tokens. The implications: JEPA-based models can learn from video (without labels) in ways that generative models can't, because predicting compressed representations is easier than predicting everything. LeCun uses this framework to argue why LLMs — which are essentially generative next-token predictors — are hitting a ceiling: they can model text well but have no grounded world model. The cake problem is the most concise summary of why Meta invests in JEPA and why LeCun is skeptical that scaling LLMs alone will reach human-level AI.