“GPT-4 will be trained on considerably more, a significant fraction of the internet as a whole.”
Our reading, the sentence that was checked:
GPT-4 will be trained on considerably more data than GPT-3, a significant fraction of the internet.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What to Expect When You’re Expecting … GPT-4, paragraph 32.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-to-expect-when-youre-expecting-19aaa4-a1
We recorded 21 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-12-25. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
From the speaker
- GPT-4 will be trained on considerably more
- GPT-3 was trained on 750
- a significant fraction of the internet as a whole
Our interpretation
- The preceding passage supplies gigabytes as the unit and GPT-3 as the comparison.
- Considerably more requires material growth, not merely exceeding 750 gigabytes by a small amount.
- Both the relative training-volume assertion and the internet-coverage assertion must hold.
- Internet coverage concerns distinct content; repeated training passes cannot be counted as additional internet coverage.
- No numerical fraction or definition of the internet is supplied, so no arbitrary percentage is imposed.
Counts as supported if: Reliable, comparable corpus measurements establish materially more GPT-4 training data than GPT-3 and a substantial share of a clearly defined, contemporaneous internet-content denominator.
Counts as contradicted if: Reliable measurements establish either no material training-data increase or only a negligible share of the relevant internet-content denominator.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
A published estimate points toward much more training data, but it does not establish the full claim. Training-token totals are not automatically measurements of unique internet content, and no comparable internet denominator was found. The evidence therefore neither supports nor contradicts the two-part assertion conclusively.
Limitations
- This is an evidence gap, not evidence that GPT-4 used little data.
- A text-only interpretation of the internet could produce a different assessment from the literal internet-as-a-whole reading.
- No deadline was stated; the unresolved issue is disclosure of a completed model's training.
Evidence (1 item)
Context only: Our World in Data,
almost 13 trillion tokens
Why it matters here: This published estimate points toward a very large training volume. It supplies neither a comparable GPT-3 byte measurement nor the unique-content numerator and internet denominator needed to settle the complete claim.
- Publisher
- Our World in Data
- Kind of source
- Secondary summary
- Describes events of
- Public from
What would settle it
A verified accounting of GPT-4's unique training corpus, a comparable GPT-3 measure, and a defensible contemporaneous internet-content denominator, with sensitivity to reasonable definitions of significant fraction.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What to Expect When You’re Expecting … GPT-4
GPT-4 will be trained on considerably more, a significant fraction of the internet as a whole.
- Where
- paragraph 32
- Published
- 2022-12-25
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-to-expect-when-youre-expecting-19aaa4-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-to-expect-when-youre-expecting-19aaa4-a1.