“It is extremely easy for a user to pose a question to an LLM, get what looks like a valid answer, and then trust to it, without doing the careful inspection necessary to check that it is actually right.”
Our reading:
Users can very easily trust plausible-looking LLM answers without checking whether they are correct.
Assessment: Supported
What happened
Was it true for the period described?
Outcome: Supported
The evidence matches what they said.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The study supports the ordinary behavioral reading: users readily accepted incorrect answers, explanations increased reliance, and many participants did not open available sources. Its verification measures make this stronger evidence than model mistakes alone. The same study found that sources and inconsistencies could reduce overreliance; Marcus did not claim that users invariably trust answers or that mitigation is impossible.
Caveats
- The experiment used selected factual questions and prepared responses in a simulated assistant.
- Extremely easy remains a qualitative judgment.
- Not opening sources is an incomplete measure of insufficient inspection.
Evidence (1 item)
Supports the claim: arXiv,
increases reliance on both correct and incorrect responses.
Why it matters: The 308-person experiment measured answer agreement and source opening; 189 participants never opened provided sources, while explanations increased acceptance of incorrect answers. Sources and visible inconsistencies reduced overreliance, qualifying the finding without eliminating the demonstrated risk.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The plain reading is that ordinary question-answering interactions readily lead users to accept plausible LLM answers without sufficient verification. It does not require every user or every answer to show this behavior.
- Acceptance of incorrect answers, accompanied by observed checking behavior or users' explanations of their decisions, tests this claim. Model error rates alone do not.
- The observation concerns user behavior documented by the post date. No starting date or reporting cycle is specified, so the target period includes earlier relevant observations rather than imposing a January–March 2025 window.
- Source clicking is an observable proxy for verification, but does not capture all careful inspection.
Right if: User studies show that participants readily accept plausible but incorrect LLM answers, with behavioral or qualitative evidence of insufficient verification in those interactions.
Wrong if: Comparable user studies consistently show that users inspect plausible answers carefully before accepting them and rarely accept incorrect answers without verification.
We wrote this test after the outcome was known.
We found 13 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Reports of LLMs mastering math have been greatly exaggerated, paragraph 3.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-04-05.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Reports of LLMs mastering math have been greatly exaggerated
It is extremely easy for a user to pose a question to an LLM, get what looks like a valid answer, and then trust to it, without doing the careful inspection necessary to check that it is actually right.
- Where
- paragraph 3
- Published
- 2025-04-05
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Supported. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.