“It is a misapprehension of LLMs to confuse the things they say with what is true.”
Our reading, the sentence that was checked:
What LLMs say should not be taken as true just because they say it; treating their statements as truth misunderstands them.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Is the US military actually afraid of Claude? A new theory of why Anthropic was labeled a supply chain risk., paragraph 7.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-is-the-us-military-actually-afraid-c9f3fa-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-03-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- LLMs
- confuse the things they say with what is true
Our interpretation
- The testable plain reading is that an LLM's assertion does not itself establish its truth. This does not mean all LLM statements are false.
- Treat the claim as describing publicly available models in the current calendar quarter through publication, January 1–March 12, 2026. Technical evaluations have no fixed reporting cycle; this reporting window is an evaluator choice.
- The assessment concerns factual reliability. It does not attempt to establish whether models have subjective experiences.
Counts as supported if: Documented evaluations within the target period identify factual assertions by available LLMs that conflict with checked facts.
Counts as contradicted if: Evidence establishes that factual assertions by the relevant LLMs reliably guarantee truth, and that the purported counterexamples are not genuine errors. Success on a limited test alone would not establish this.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
A dated evaluation records false factual assertions from an available LLM, checked against the subject's biography. That supports the ordinary claim that saying something does not establish its truth. Other models answered this particular prompt without detected errors; this does not make model assertions inherently authoritative.
Limitations
- The benchmark uses one person's biography and that person's manual judgments; it cannot establish a general error rate.
- The cited report was published after the post but records tests within the target period.
- This evidence does not settle consciousness, anxiety, or the accuracy of every model self-report.
Evidence (1 item)
Supports the claim: Martin Monperrus,
while others generate multiple false claims.
Why it matters here: Table 2 records eight hallucinations for moonshotai/kimi-k2 on February 1, checked by the biographical subject. The same day's Claude Opus 4.5 result has zero, showing that the finding concerns fallibility rather than universal failure.
- Publisher
- Martin Monperrus
- Kind of source
- Benchmark result
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Is the US military actually afraid of Claude? A new theory of why Anthropic was labeled a supply chain risk.
It is a misapprehension of LLMs to confuse the things they say with what is true.
- Where
- paragraph 7
- Published
- 2026-03-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-is-the-us-military-actually-afraid-c9f3fa-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-is-the-us-military-actually-afraid-c9f3fa-a1.