“Even with thinking mode activated, the output was filled with errors that could easily have been avoided, with a 5-second Wikipedia search that any sensible human could have performed.”
Our reading, the sentence that was checked:
GPT-5 in thinking mode produced a list of Canadian Prime Ministers full of easily avoidable errors.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, LLMs are not like you and me—and never will be., paragraph 14.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-llms-are-not-like-you-and-meand-never-cee75f-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- thinking mode activated (qualifier, stated by the speaker)
- Canadian PM list task (qualifier, stated by the speaker)
From the speaker
- thinking mode activated
- the output was filled with errors
- a 5-second Wikipedia search
- Jean Chrétien appears twice
- William Lyon McKenzie King died before he was born
- a fictional Kim Camphe
Our interpretation
- This concerns the particular Canadian prime minister image described in the post, not GPT-5's general accuracy.
- Filled with errors means multiple substantive factual errors in that output.
- The five-second comparison means the correct information was readily accessible; it is not treated as a measured human completion time.
- The target period is August 2025 through publication. The surrounding account describes a recent replication but does not establish its generation date.
- Attribution requires evidence distinguishing the conversational model, its thinking mode, and any image-generation tool.
Counts as supported if: An authenticated conversation establishes that the referenced output was produced through GPT-5 with thinking activated, and comparison with historical records confirms multiple substantive errors.
Counts as contradicted if: An authenticated record establishes that the referenced output lacked the alleged errors, or that its attribution to GPT-5 with thinking activated was incorrect.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The searches located Marcus's account and a later mention of the Canadian image, but no authenticated original conversation. That leaves the specific output and thinking-mode attribution unverified. This does not show that Marcus's description was false.
Limitations
- The retrieved plain-text version of the post does not supply independently verifiable image provenance.
- A new reproduction would test present behavior, not authenticate the reported incident.
Evidence (0 items)
No evidence item settled this claim by the cutoff.
What would settle it
The original conversation or authenticated export, including the image and model, mode, and tool metadata, followed by comparison with independent historical records.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, LLMs are not like you and me—and never will be.
Even with thinking mode activated, the output was filled with errors that could easily have been avoided, with a 5-second Wikipedia search that any sensible human could have performed.
- Where
- paragraph 14
- Published
- 2025-08-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-llms-are-not-like-you-and-meand-never-cee75f-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-llms-are-not-like-you-and-meand-never-cee75f-a1.