“This kind of flaw, in which reasoning fails in light of distracting material, is not new.”
Our reading:
The failure of language model reasoning when distracting material is present is a long-known flaw, documented as early as 2017.
Assessment: Supported
What happened
Was it true for the period described?
Outcome: Supported
The evidence matches what they said.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
Jia and Liang’s 2017 study used AddSent to add distracting sentences designed to preserve the correct answer [e1]. Average F1 fell from 75.7% to 31.3% across four tested reading-comprehension models [e1]. These results support the historical claim that comparable failures under distracting text were documented in 2017 [e1].
Caveats
- The study measured reading comprehension in 2017 systems. It does not establish a shared failure mechanism with modern LLMs or an inability to perform formal reasoning.
- The answer-preservation method applies to AddSent. The separate AddAny method did not guarantee that additions preserved the correct answer.
Evidence (1 item)
Supports the claim: arXiv,
accuracy drop under every form of adversarial evaluation.
Why it matters: The authors constructed distracting sentences to preserve the correct answer and reported that AddSent reduced average F1 from 75.7% to 31.3% across four tested systems. This directly documents the comparable failure Marcus identifies in 2017.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The plain reading is that comparable failures under distracting text had already been documented in language-understanding systems in 2017.
- Comparable means that adding distracting text reduced correct answers without changing the answer the original passage supported. It does not require identical architectures, tasks, or underlying mechanisms.
- The target period is 2017 because the surrounding text explicitly identifies that year's Jia–Liang study.
Right if: The 2017 Jia–Liang study reports that adding distracting sentences that preserve the correct answer substantially worsened reading-comprehension model performance.
Wrong if: The cited study found no such deterioration, or its reported failures depended on changing the correct answer rather than adding distracting material.
We wrote this test after the outcome was known.
We found 10 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, LLMs don’t do formal reasoning - and that is a HUGE problem, paragraph 6.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2024-10-11.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, LLMs don’t do formal reasoning - and that is a HUGE problem
This kind of flaw, in which reasoning fails in light of distracting material, is not new.
- Where
- paragraph 6
- Published
- 2024-10-11
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Supported. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
The final check failed: a cited source didn’t say what the write-up said, and the write-up said more than the evidence showed, and our summary didn’t match their words.