LLMs' cognitive capacities are severely limited, as shown by their errors, hallucinations, and inability to fact-check their own work.

Not graded on evidence

Outcome: Can’t be checked

An opinion, a recommendation, or too vague to test. Listed, not scored.

Evidence up to 2026-10-28. Checked 2026-10-31.

Why

Marcus's main claim is a broad judgment that the 'cognitive capacities' of LLMs are 'severely limited'. He names no task, model, measure or threshold. The resolution contract finds no agreed standard for that phrase. One of his stated reasons is that LLMs cannot fact-check their own work. An ICLR 2024 paper, in a version dated 14 March 2024, reports that LLMs struggle to correct their own reasoning without external feedback. It also reports that their performance sometimes gets worse after self-correction (e1). OpenAI's GPT-4 Technical Report was first released in March 2023, and its cited version is dated 4 March 2024. It says GPT-4 is less capable than humans in many real-world scenarios. It also says GPT-4 shows human-level performance on various professional and academic benchmarks, including a simulated bar exam score around the top 10% of test takers (e2). Whether these findings amount to 'severely limited' depends on a standard the claim does not give. So the claim cannot be scored as supported or contradicted.

Caveats

• e1 covers only intrinsic self-correction on reasoning tasks. It does not test hallucination rates or 'cognitive capacity' in general.
• e1 reports that LLMs 'struggle' to self-correct. It does not show whether that amounts to the 'utter inability' in Marcus's wording.
• e2 covers a single model, GPT-4. It is the developer's own disclosure, not an independent evaluation.
• The narrower premises (errors, hallucinations, weak self-checking) were not graded separately.
• The evidence packet has only two items, and both are context. Neither tests the broad judgment directly.

Evidence (2 items)

Claims like this aren’t scored, so there is no evidence to show.