GPT-5 and irrational exuberance
We recorded 17 claims from this post. 11 were checked against evidence. 6 can’t be checked. 9 more are held. Read the post at its source.
Claims from this post (17)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Rhetoric
“I am not afraid because GPT is too smart, I am afraid because GPT is too stupid, too dumb to comprehend an op-ed, and too dumb to keep its mouth shut.”
Our reading: Marcus says he is afraid not because GPT is too smart but because it is too stupid: too dumb to comprehend an op-ed and too dumb to keep its mouth shut.
No horizon
Assessment: SupportedSaid
Observation
“It’s not smart enough to filter out falsehood, but just smart enough to be dangerous, creating and spreading falsehoods it fails to verify.”
Our reading: GPT cannot filter out falsehood and creates and spreads falsehoods it fails to verify, making it dangerous.
No horizon Evidence as of
Limitation: Volokh's unsuccessful searches support his assessment that the examples were fabricated, but are not exhaustive proof that no corresponding source exists [e2].
Assessment: UnresolvedSaid
Forecast
“GPT-5 is not going to be magic. It will not give China, the US, or anyone else interstellar starships; it will not give anyone an insuperable military advantage.”
Our reading: GPT-5 will not give China, the US, or anyone else interstellar starships, and it will not give anyone an insuperable military advantage.
No deadline stated Evidence as of
Limitation: Military capabilities may not be publicly observable.
Assessment: UnresolvedSaid
Observation
“GPT-4 can’t even play a good game of chess (it’s about as good as chess computers from 1978).”
Our reading: GPT-4 cannot play a good game of chess and is about as good as 1978 chess computers.
No horizon Evidence as of
Limitation: The admitted GPT-4 evidence consists of one recorded game and an account of two attempted games [e1, e2].
Assessment: UnresolvedSaid
Forecast
“It might be on the nice-to-have list; it will not mark the mythical singularity.”
Our reading: GPT-5 will not mark the singularity.
No deadline stated Evidence as of
Limitation: The cited failure rate concerns one task, not a measurement of sustained, self-amplifying intelligence improvement [e1].
Assessment: UnresolvedSaid
Forecast
“It will be even better than GPT-4 at creating convincing sounding prose.”
Our reading: GPT-5 will be better than GPT-4 at creating convincing-sounding prose.
No deadline stated Evidence as of
Limitation: Writing preference varies with genre, prompting, and generation settings.
Assessment: UnresolvedSaid
Observation
“(Not one of my predictions about GPT-4 proved to be incorrect; every flaw I predicted would persist persisted.)”
Our reading: Marcus says none of his predictions about GPT-4 proved incorrect, and every flaw he predicted would persist in GPT-4 did persist.
No horizon Evidence as of
Limitation: Confirming several central predictions cannot establish a universal track-record claim.
Assessment: UnresolvedSaid
Forecast
“Their forte has been and always will be making shit up; they can’t for the life of them (speaking metaphorically of course) check facts.”
Our reading: For GPT models on their own, making things up has been and always will be their forte, and they cannot check facts.
No deadline stated Evidence as of
Limitation: A literal zero-ability reading would yield a different assessment of the fact-checking component.
Assessment: UnresolvedSaid
Conditional forecast
“Some form of AI may eventually do everything people are imagining, revolutionizing science and technology and so on, but LLMs will be at most only a tiny part of whatever as-yet-uninvented technology does that.”
Our reading: If some as-yet-uninvented form of AI eventually does everything people are imagining, such as revolutionizing science and technology, LLMs will be at most only a tiny part of it.
No deadline stated Evidence as of
Only if: if some form of AI eventually does everything people are imagining
Assessment: Can’t be checkedSaid
Observation
“What’s happening now, in my view, is that people are wildly overattributing an intelligence to GPT-4 that isn’t there, and then scaling up that further into something that is some kind of total fantasy about GPT-5.”
Our reading: People are wildly overattributing intelligence to GPT-4 and extrapolating that into fantasies about GPT-5.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“A lot of those fantasies get started when people don’t really understand what’s happening with GPT-4; over and over they misattribute occasionally startling feats of text manipulation over vast corpora for real understanding.”
Our reading: A lot of the fantasies about GPT-5 get started when people don't really understand GPT-4 and repeatedly mistake its occasionally startling feats of text manipulation over vast corpora for real understanding.
No horizon
Scope: a lot of those fantasies
Assessment: UnresolvedSaid
Observation
“And very little of what GPT-4 does is fully replicable. Sometimes summarization (aka compression) works (super cool!) and sometimes it doesn’t, as many were quick to point out:”
Our reading: Very little of what GPT-4 does is fully replicable; its summarization sometimes works and sometimes fails.
No horizon Evidence as of
Limitation: The task population and meaning of full replication are unspecified.
Assessment: ContradictedSaid
Observation
“Essentially none of the fancy cognitive things GPT-4 is rumored to do stand up.”
Our reading: Essentially none of the advanced cognitive abilities attributed to GPT-4 hold up under scrutiny.
No horizon Evidence as of
Limitation: The result depends on counting controlled reasoning performance among the unspecified 'fancy cognitive things'.
Assessment: SupportedSaid
Observation
“Maarten Sap from Yejin Choi’s lab dove deeper, with a more thorough investigation, and confirmed what we suspected: the claim that large language models have mastered theory of mind is a myth:”
Our reading: The claim that large language models have mastered theory of mind is a myth.
No horizon Evidence as of
Limitation: The studies test particular models, tasks and prompts, rather than every aspect of theory of mind [e5, e6].
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We in the West should definitely be concerned about whether China is going to catch up over the next decade. But basing our AI policies around fantasies doesn’t seem like a particularly good idea.”
Our reading: The West should be concerned about China catching up over the next decade but should not base AI policy on fantasies about GPT-5.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We should be super worried about how large language models, which are very difficult to steer, are changing our world, but the immediate transition to a singularity is one worry we can postpone.”
Our reading: We should worry a great deal about how hard-to-steer LLMs are changing the world, but not about an immediate transition to a singularity.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“If China wins the AI war it will be partly because of data, partly because of choices about how to allocate resources, and partly because they ultimately do a better job educating their citizens and become the first to invent something new, and not because they scale up the exciting but rickety contraption that’s currently obsessing the West.”
Our reading: If China wins the AI war, it will be partly because of data, partly because of choices about allocating resources, and partly because it ultimately does a better job educating its citizens and becomes the first to invent something new, not because it scales up the exciting but rickety contraption currently obsessing the West.
No horizon
Only if: if China wins the AI war
Held from this post (9)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: GPT cannot be used as a reliable data processor, won't drive cars reliably, and is not going to be the right tool for tracking air forces in flight.
- The record does not meet the current rules. Our reading, not assessed: To Marcus, the idea that GPT-5 will fundamentally transform the world into some sort of new AGI regime sometime in Q4 2023, for China or the US, still seems like a fantasy.
- The record does not meet the current rules. Our reading, not assessed: GPT-5 will not be smart enough to revolutionize fusion or nanotech, and starships will not arrive in 2024.
- The criteria are ambiguous. Our reading, not assessed: GPT models regurgitate ideas rather than inventing them.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: GPT-5 will have some impact on defense and security.
- The criteria are ambiguous. Our reading, not assessed: GPT models on their own do not do scientific discovery.
- The record does not meet the current rules. Our reading, not assessed: It is a safe bet that GPT-5 will be a lot like GPT-4, doing the same kinds of things and having the same kinds of flaws, but being somewhat better.