There is still no robust solution to LLM hallucinations.

Was it true for the period described?

Outcome: Supported

The dated evidence matches what the claim committed to, under the criteria frozen before grading.

Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.

Evidence as of

2026-10-28

Deadline state when assessed

No horizon

Assessment recorded

2026-10-29

Policy version

2026-10-28.7

Rationale

The claim describes conditions in 2024, up to the post date of 1 January 2025. On 7 November 2024, OpenAI researchers wrote that current frontier models sometimes produce false outputs, known as hallucinations. They also called training models to give factually correct answers an open problem (e1). The same paper introduced SimpleQA, a set of short fact-seeking questions, and said GPT-4o and Claude both scored under 50% on it (e2). None of the cited evidence shows a method that reliably removed hallucinations across varied tasks by the end of 2024. Under the contract, a lab saying in late 2024 that hallucination was unsolved counts as support, so the claim holds on a plain reading.

Limitations

• The main source is a lab paper describing its own models' limits. Independent leaderboard figures for late 2024 could not be read.
• A SimpleQA score under 50% is not the same as a hallucination rate, because the benchmark also counts questions the model did not attempt.
• The evidence does not measure hallucination rates in grounded settings such as retrieval or summarization.
• 'Robust' has no set threshold.

Evidence (2 items)

• Supports the claim: OpenAI (arXiv 2411.04368), 2024-11-07

  Current frontier models sometimes produce false outputs or answers that are not substantiated by evidence, a problem known as “hallucinations.”

  Why it matters here: OpenAI researchers said in November 2024 that frontier models still hallucinate, and the paper calls factual accuracy an open problem.

  Where

  section 1 Introduction, paragraph 1

  Publisher

  OpenAI (arXiv 2411.04368)

  Kind of source

  Preprint, not peer reviewed

  Describes events of

  2024-11-07

  Public from

  2024-11-07

• Supports the claim: OpenAI (arXiv 2411.04368), 2024-11-07

  SimpleQA is created to be challenging for frontier models (e.g., GPT-4o and Claude both score less than 50%).

  Why it matters here: Leading models answered fewer than half of simple fact questions correctly at the end of 2024. That does not look like a robust fix for false answers.

  Where

  section 1 Introduction, bullet 'Challenging for frontier models'

  Publisher

  OpenAI (arXiv 2411.04368)

  Kind of source

  Preprint, not peer reviewed

  Describes events of

  2024-11-07

  Public from

  2024-11-07