Chollet on X: Abstraction vs. Compression

Chollet draws a precise distinction between compression (finding compact representations of observed data) and abstraction (building executable programs that generalize to unobserved cases). LLMs are remarkable compressors — they encode vast amounts of human knowledge in compressed neural representations. But abstraction requires something more: the ability to construct new programs, not just recall compressed versions of old ones. ARC tasks test specifically for abstraction, not compression. This distinction has implications for interpretability research: interpreting an LLM's internal representations is studying compression; understanding AGI requires studying abstraction.