Chollet's Response to 'But ChatGPT Can Do This'

A thread addressing the common argument that because ChatGPT can perform impressively on various tasks, it must be approaching AGI. Chollet argues that task performance on training-distribution tasks is not evidence of general intelligence — it is evidence of the quality of training data and scale. He uses the analogy of a well-stocked library: a library can "answer" many questions correctly, but the library itself isn't intelligent. The intelligence (if any exists) was that of the writers who contributed to it. LLMs are very efficient libraries, but libraries nonetheless. The test for AGI is whether the "library" can write new books in domains it has never seen — which is exactly what ARC tests.