Noam Chomsky and GPT-3
We recorded 8 claims from this post. 2 were checked against evidence. 6 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (8)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Causal claim
“Human minds try to relate their language to what they know about the world; large language models don’t. Which means the latter just aren’t telling us much about the former.”
Our reading: Because large language models, unlike human minds, do not try to relate their language to what they know about the world, they just aren't telling us much about human minds.
No horizon
Assessment: UnresolvedSaid
Causal claim
“There is no intention to create misinformation—but also no capacity to avoid it, because fundamentally, GPT-3 is a model of how words relate to one another, not a model of how language might relate to the perceived world.”
Our reading: GPT-3 cannot avoid producing misinformation because it models how words relate to each other, not how language relates to the world.
No horizon Evidence as of
Limitation: The evidence is about error rates, not about mechanism.
Assessment: Can’t be checkedSaid
Causal claim
“Systems like GPT-3 don’t tell us why human languages have the special character that they do. As such, there is little explanatory value.”
Our reading: Because systems like GPT-3 don't tell us why human languages have the special character that they do, they have little explanatory value.
No horizon
Scope: systems like GPT-3
Assessment: ContradictedSaid
Forecast
“They do not gravitate towards human language, and they never develop a cohesive undestanding of the world, no matter how much experience they have.”
Our reading: Systems like GPT-3 do not gravitate towards human language, and they never develop a cohesive understanding of the world, no matter how much experience they have.
Settled before its deadline Evidence as of
Limitation: The contradiction applies to the language-learning component of the combined assertion.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“They are an embarrassment to Locke’s conjecture, and a strong argument for exploring their very antithesis, which would be Kantian systems that start with representations of space and time and causality, and at least some kernels of universal grammar.”
Our reading: LLMs' failures argue strongly for exploring Kantian AI systems that start with built-in representations of space, time, causality and some universal grammar.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Humans know these things; if AI’s are to understand language, they should refract language through this sort basic understanding of common sense, rather than needlessly starting as blank slates and acquiring statistics without concepts.”
Our reading: For AI to understand language, it should process language through built-in commonsense understanding instead of starting as a blank slate and learning statistics.
No horizon
Qualifier: if AI's are to understand language
Assessment: Can’t be checkedSaid
Forecast
“A new breed of AI systems, with a firmer innate basis, closer to the Kantian model, but out of step with current Zeitgeist, might eventually tell us more.”
Our reading: New AI systems with a firmer innate, Kantian basis might eventually tell us more about the human mind.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“Large language models can, parasitically, glom onto human data, but they could glom onto any other data set just as well (e.g. programming languages); as such, they tell us far too little about how humans have come to be what they are.”
Our reading: Because large language models could glom onto any other data set (e.g. programming languages) just as well as human data, they tell us far too little about how humans have come to be what they are.
No horizon
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: GPT-3's prediction is basically regurgitation of training patterns and does not mean it knows what the words it uses mean.