Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Gary Marcus Observation First said

“Now, as far as I know, LLM hallucinations are still a serious problem.”

Promises are cheap, paragraph 5.

Our reading:

As far as he knows, LLM hallucinations are still a serious problem.

Assessment: Can’t be checked

What happened

Not graded on evidence

Outcome: Can’t be checked

An opinion, a recommendation, or too vague to test. Listed, not scored.

Evidence up to . Checked .

Why

Marcus explicitly qualifies this assessment with “as far as I know”. The earlier proposed test would replace that wording with a firm claim about substantial hallucination rates. Under the instruction excluding hedged assessments from the plain-reading allowance, that substitution is not justified. The claim is therefore not adjudicable as written; this is not a finding that hallucinations were harmless or resolved.

Caveats

  • No external evidence search was undertaken because the contract cannot fairly define a scored outcome.
  • This assessment does not grade the separate statements about lawyers or documented legal cases.

Evidence (0 items)

Claims like this aren’t scored, so there is no evidence to show.

How we judged it

Kind of claim
Observation
Deadline
None given. We don’t add one.

Conditions

  • as far as I know (their words)

Where we had to interpret

  • The qualification applies to the whole assessment and must remain part of the claim.
  • The speaker gives a qualified assessment, without specifying a measure of seriousness. Turning it into a firm claim about substantial benchmark error rates would add a commitment he did not make.
  • “As far as I know” is an epistemic qualification, not a conditional forecast antecedent.

We don’t grade claims like this. We only check the quote and who said it.

We found 10 claims in this post. See them all.

Full record

Source and dates

Marcus on AI, Promises are cheap, paragraph 5.

Said
We read the source
live page, full text, public
Evidence up to
Checked

We read this wording on 2026-10-28. The post may have been edited after 2026-02-12.

Where it was said (1)

Said once in the posts we read.

  • : Marcus on AI, Promises are cheap

    Now, as far as I know, LLM hallucinations are still a serious problem.

    Published
    2026-02-12
    We read it
    , live page
    Access
    full text, public

History

Versions

  • Current: Can’t be checked. Evidence up to 2026-10-28.

What happened to this record

  1. PublishedBy us

    Published after both AI reviews and the final check passed.

  2. Held backBy us

    A reviewer said our test wasn’t fair to what was actually said.