“I can’t think of a single measure in which any existing model in 2024 outpace GPT-4 by as much as GPT-4 outpaced the state of the art in 2020. We are already behind the curve.”
Our reading:
Marcus says he cannot think of a single measure on which any model existing in 2024 outpaces GPT-4 by as much as GPT-4 outpaced the 2020 state of the art, and says progress is already behind the curve.
Assessment: Can’t be checked
What happened
Not graded on evidence
Outcome: Can’t be checked
An opinion, a recommendation, or too vague to test. Listed, not scored.
Evidence up to . Checked .
Why
On June 5, 2024, Marcus said he could not think of a qualifying measure for models existing by that date [e1]. This reports his recollection; it does not assert unconditionally that no such measure exists [e1]. Benchmark results therefore cannot settle the statement as worded. His accompanying judgment that progress was “behind the curve” supplies no defined measurable target [e1]. The claim is not adjudicable as written.
Caveats
- The sole evidence item establishes Marcus’s wording and context, not comparative model performance [e1].
- This assessment does not establish whether benchmark gains had slowed or whether a qualifying measure existed.
Evidence (1 item)
Claims like this aren’t scored, so there is no evidence to show.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- I can't think of (their words)
- models existing in 2024 as of the post (their words)
Where we had to interpret
- The first sentence reports Marcus’s qualified recollection as of June 5, 2024. It does not unconditionally assert that no qualifying benchmark exists.
- The comparison concerns models existing by the post date, not models released later in 2024.
- The final sentence expresses his assessment of progress against the trajectory discussed in the post. He supplies no measurable target for being behind that trajectory.
- Selecting benchmarks, a definition of improvement, or a numerical trajectory would add commitments that he did not make.
We don’t grade claims like this. We only check the quote and who said it.
We found 8 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, AGI by 2027?, paragraph 8.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2024-06-05.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, AGI by 2027?
I can’t think of a single measure in which any existing model in 2024 outpace GPT-4 by as much as GPT-4 outpaced the state of the art in 2020. We are already behind the curve.
- Where
- paragraph 8
- Published
- 2024-06-05
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Can’t be checked. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.