Caricaturing Noam Chomsky
We recorded 5 claims from this post. 4 were checked against evidence. 1 can’t be checked. 6 more are held. Read the post at its source.
Claims from this post (5)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“Still, even though Chomsky’s argument surely could have used considerably more nuance (see my article with Ernest Davis on how not to evaluate GPT), his overall point is correct: LLMs don’t reliably understand the world.”
Our reading: LLMs do not reliably understand the world.
No horizon Evidence as of
Limitation: It rests on a behavioural reading of 'understand'. Someone who reads 'understanding' differently could dispute it.
Assessment: UnresolvedSaid
Observation
“Minerva, for example, is trained on an immense amount of data, and ostensibly captures two-digit arithmetic, but it fails altogether on 4-digit multiplication,”
Our reading: Minerva, despite immense training data and apparent two-digit arithmetic ability, fails altogether on 4-digit multiplication.
No horizon Evidence as of
Limitation: The decisive data exist only in a figure that the text tools cannot read.
Assessment: UnresolvedSaid
Observation
“The same goes for essentially everything an LLMs talks about; there is a degree of generalization to similar examples, but distribution shift is hard (the crux of my own work going back to 1998), and nearly any generalization can be easily broken.”
Our reading: For essentially everything an LLM talks about, there is a degree of generalization to similar examples, but distribution shift is hard and nearly any generalization can be easily broken.
No horizon Evidence as of
Limitation: All the evidence comes from math and numerical tasks, not the range of domains the claim covers.
Assessment: UnresolvedSaid
Observation
“The request is perfectly clear, not a nonsensical question in any way. The prompt is perfectly sensible; the system just isn’t up to the job.”
Our reading: The request in his example is perfectly clear and sensible, not nonsensical in any way, and the system just isn't up to the job.
No horizon Evidence as of
Limitation: The retrieval gap is not evidence that the prompt was unclear or that the response was correct.
Assessment: Can’t be checkedSaid
Observation
“But there is no real argument against Chomsky’s overall take: these systems are useful technology but remain a long way from true artificial intelligence, and they are even further from telling us anything useful about the function of the human mind.”
Our reading: LLMs are useful technology but remain a long way from true artificial intelligence and even further from telling us anything useful about the human mind.
No horizon
Held from this post (6)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: What Marcus said and what Chomsky said is correct: the system (ChatGPT) really isn't up to the job.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: LLMs certainly have not taught us anything whatsoever about why the world is as it is rather than some other way, and the same goes for why the human mind is as it is.
- The two reviews reached different assessments. Our reading, not assessed: ChatGPT is wildly stochastic and unreliable, so single examples of its behavior prove little.
- The criteria are ambiguous. Our reading, not assessed: Minerva's arithmetic failures are most parsimoniously explained by pattern recognition over stored examples rather than genuine understanding of multiplication.