Multi-Modal Intelligence: Why Video Is More Important Than Text

Data2Vec is Meta's first paper demonstrating a unified self-supervised learning framework that works across vision, language, and speech with a single architecture and objective. The core idea: rather than predicting pixels, tokens, or spectrograms, predict the top-layer representations of the full input from a masked version. This is a form of self-distillation and an early form of the JEPA objective applied across modalities. The paper is important for understanding LeCun's multimodal vision: intelligence requires integrating signals from multiple modalities because no single modality is sufficient to build a complete world model. Text tells you about social and linguistic structures. Images tell you about visual categories. Video tells you about physics, causality, and how objects move in the world. Audio tells you about language prosody and environmental events. A system that learns from all modalities simultaneously — predicting representations across modalities rather than raw data in any single modality — will learn a richer, more grounded world model than any single-modality system. Data2Vec was a stepping stone to V-JEPA and I-JEPA, and represents Meta's approach to multimodal SSL before the LLM wave made text-only approaches dominate AI investment.