Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
We recorded 10 claims from this post. 2 were checked against evidence. 8 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“I do not share his confidence.”
Our reading: Marcus does not share Sam Altman's confidence that OpenAI knows how to build AGI.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Here is a thread of links to a several recent observations that I think show how far we are from robust intelligence.”
Our reading: Marcus thinks several recent observations show how far AI is from robust intelligence.
No horizon
Qualifier: I think
Assessment: Can’t be checkedSaid
Observation
“o1 probably isn’t as robust as people might have hoped. Even on certain benchmarks, results may be fragile or difficult to replicate:”
Our reading: o1 is probably less robust than hoped, and some benchmark results may be fragile or hard to replicate.
No horizon
Assessment: Can’t be checkedSaid
Observation
“In domains where that’s not possible, o1 may not even be that much better than GPT-4 (e.g on some language tasks), as you can see if you read OpenAI’s reports carefully.”
Our reading: Where data augmentation isn't possible, o1 may not be much better than GPT-4, for example on some language tasks.
No horizon
Qualifier: in domains where data augmentation isn't possible
Assessment: Can’t be checkedSaid
Observation
“In o1’s best performances cases, there may have been heavy data augmentation that is not possible in more open-ended domains.”
Our reading: In o1's best-performance cases there may have been heavy data augmentation that is not possible in more open-ended domains.
No horizon
Scope: in o1's best-performance cases
Assessment: Can’t be checkedSaid
Forecast
“The presentation of o3 was problematic, and no clean test of out-of-domain generalization was provided, suggesting that we may see the kinds of problems with distribution shift as ever, outside of semi-closed domains where synthetic data can readily be generated.”
Our reading: Marcus says the presentation of o3 was problematic and gave no clean test of out-of-domain generalization, which suggests o3 may show the same distribution-shift problems as ever outside semi-closed domains where synthetic data can readily be generated.
No horizon
Scope: outside semi-closed domains where synthetic data can readily be generated
Assessment: Can’t be checkedSaid
Forecast
“[Note also that the current idea, test-time “scaling”, is very expensive, and will likely prove unreliable out of domains where augmentation is adequate, in which case it will get comparatively little commercial traction.”
Our reading: Test-time scaling is very expensive and will likely prove unreliable outside domains where data augmentation is adequate.
No horizon
Qualifier: outside domains where augmentation is adequate
Assessment: UnresolvedSaid
Observation
“Even o1 is losing money; o3 isn’t even available, and perhaps cannot be run at a profit.]”
Our reading: OpenAI's o1 is losing money.
No horizon Evidence as of
Limitation: The evidence covers only the ChatGPT Pro tier. It does not cover o1 sold through other channels, such as ChatGPT Plus or the API.
Assessment: UnresolvedSaid
Causal claim
“As I noted in 2001, the lack of distinct, accessible, reliable database-style records leads to hallucinations.”
Our reading: The lack of distinct, accessible, reliable database-style records leads to hallucinations, as Marcus noted in 2001.
No horizon Evidence as of
Limitation: The historical search was not an exhaustive examination of every 2001 publication.
Assessment: Can’t be checkedSaid
Observation
“But so far I have not seen a principled solution to even one of these longstanding problems, and that tells me a lot.”
Our reading: So far Marcus has not seen a principled solution to even one of these longstanding problems, and he says that tells him a lot.
No horizon
Qualifier: so far
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: If test-time scaling proves unreliable outside domains where augmentation is adequate, it will get comparatively little commercial traction.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Despite many promises to the contrary, the lack of reliable database-style records and the hallucinations it causes still routinely lead to inaccurate news summaries, defamation, fictional sources, incorrect advice, and unreliability.