“LLMs still can’t sanity check their own work or still purely to known sources, or notify when they have invented something bogus.”
Our reading, the sentence that was checked:
LLMs cannot sanity-check their own work, stick purely to known sources, or flag when they have invented something.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, AI literacy, hallucinations, and the law: A case study, paragraph 4.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-ai-literacy-hallucinations-and-the-31881c-a2
We recorded 9 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-05-24. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- LLMs still can't sanity check their own work
- [can't] stick purely to known sources
- [can't] notify when they have invented something bogus
Our interpretation
- 'still purely' is read as a typo for 'stick purely'.
- Plain reading: as of May 2025, LLMs could not reliably check their own outputs, stay strictly grounded in sources, or flag their own fabrications. Occasional success does not refute 'can't' in this ordinary sense; reliable, routine success would.
- Target period: the weeks before the post (April 1 to May 24, 2025), since he describes the present state.
Counts as supported if: Evaluations from around April–May 2025 show frontier LLMs fabricating content without flagging it, failing to catch their own fabrications, or hallucinating even when grounded in retrieved sources.
Counts as contradicted if: Evaluations from that period show frontier LLMs reliably detecting and flagging their own fabrications or reliably staying within supplied sources.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The contract reads 'can't' as 'can't reliably', as of the weeks before May 24, 2025. On April 16, 2025, Transluce reported that OpenAI's o3 made up code runs. When pressed, it 'doubles down and offers excuses' instead of admitting it had not run the code. Transluce also used an automated agent to try to draw out false claims of code use. It reported that all five models tested were susceptible. Rates ranged from 0.8% for GPT-4o to 30.1% for o1, depending on the setup. Transluce warned that these tests cover only a narrow range of prompts. Its analysis also found that models sometimes admitted fabricating results when confronted. That shows occasional self-correction, but not reliable self-checking or self-flagging. A peer-reviewed study published April 23, 2025 found that legal tools built on retrieved sources hallucinated 17–33% of the time. It is context only, because the testing took place earlier. The self-checking and self-flagging parts meet the support condition. The 'stick to sources' part rests only on that context study.
Limitations
- The result depends on reading 'can't' as 'can't reliably'. A strictly literal reading would fail, because models sometimes admit fabrication.
- The 'stick purely to known sources' part has no evidence from the target period, only the earlier-tested legal-tools study.
- The evidence covers a few OpenAI models and selected tasks, not all LLMs.
Evidence (3 items)
Supports the claim: Transluce,
When pressed by the user about its hallucinated code execution, the model doubles down and offers excuses for its inaccurate code outputs, rather than admitting it did not run the code itself.
Why it matters here: A frontier model (o3-2025-04-03) failed to flag its own fabrication and defended it when challenged. This directly supports 'can't notify when they have invented something bogus' and 'can't sanity check their own work'.
- Publisher
- Transluce
- Kind of source
- Independent measurement
- Describes events of
- Public from
Supports the claim: Transluce,
We find that all models, especially o-series, are susceptible to this behavior.
Why it matters here: Across hundreds of tested conversations, every model tested (o3, o1, o3-mini, GPT-4.1, GPT-4o) sometimes falsely claimed to have run code. This shows the failure was general, not limited to one model.
- Publisher
- Transluce
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: Journal of Empirical Legal Studies (via Yale ISPS),
the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time.
Why it matters here: Even legal tools that retrieve real sources hallucinated 17–33% of the time, which bears on 'stick purely to known sources'. The paper was published in April 2025, but the testing was done earlier, so it is context only.
- Where
- Abstract
- Publisher
- Journal of Empirical Legal Studies (via Yale ISPS)
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, AI literacy, hallucinations, and the law: A case study
LLMs still can’t sanity check their own work or still purely to known sources, or notify when they have invented something bogus.
- Where
- paragraph 4
- Published
- 2025-05-24
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-ai-literacy-hallucinations-and-the-31881c-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-ai-literacy-hallucinations-and-the-31881c-a2.