Adding visual data to LLMs has not solved hallucinations.

Was it true for the period described?

Outcome: Supported

The dated evidence matches what the claim committed to, under the criteria frozen before grading.

Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.

Evidence as of

2026-10-28

Deadline state when assessed

No horizon

Assessment recorded

2026-10-29

Policy version

2026-10-28.7

Rationale

The claim is that, as of April 2024, adding visual data to LLMs had not solved hallucinations. OpenAI's system card for GPT-4V, dated 25 September 2023, says the model 'was prone to hallucinations' and sometimes made factual errors in an authoritative tone (e1). HallusionBench, accepted to CVPR 2024, tested 15 vision-language models, including GPT-4V, Gemini Pro Vision and Claude 3, in a version dated 25 March 2024. GPT-4V scored highest, with 31.42% question-pair accuracy, and all other models scored below 16%. The paper names language hallucination as one of the failure modes it observed (e2). Both sources fall within the target period, April 2023 to 15 April 2024. Together they show that leading multimodal LLMs still hallucinated in material ways, which meets the support condition. A survey posted on 29 April 2024, after the period ended, says multimodal LLMs often produce hallucinations (e3). It is context only.

Limitations

• 'Solved' is not defined. The claim is read as hallucinations remaining a material problem, not as adding vision bringing no improvement at all. The evidence packet notes that the GPT-4V system card reports fewer hallucinations during a beta, but that passage is not in the cited excerpt.
• HallusionBench reports question-pair accuracy on a set of 346 images and 1,129 questions written by experts. It does not measure how often hallucinations happen in everyday use.
• The evidence covers a small number of sources. It does not show how often every multimodal model released in the period hallucinated.

Evidence (3 items)

• Supports the claim: OpenAI, 2023-09-25

  Additionally, the model was prone to hallucinations and sometimes could make factual errors in an authoritative tone.

  Why it matters here: The developer of GPT-4V, the main model that added vision to an LLM, reports that it was still prone to hallucinations.

  Where

  Section on scientific proficiency, page 6

  Publisher

  OpenAI

  Kind of source

  Company disclosure: the company describing itself, not an independent finding

  Describes events of

  2023-09-25

  Public from

  2023-09-25

• Supports the claim: arXiv (accepted to CVPR 2024), 2024-03-25

  we benchmarked 15 different models, highlighting a 31.42% question-pair accuracy achieved by the state-of-the-art GPT-4V. Notably, all other evaluated models achieve accuracy below 16%.

  Why it matters here: A benchmark of leading vision-language models, including GPT-4V, Gemini Pro Vision and Claude 3, found low accuracy and documented hallucination failure modes weeks before the post.

  Where

  Abstract

  Publisher

  arXiv (accepted to CVPR 2024)

  Kind of source

  Peer-reviewed paper

  Describes events of

  2024-03-25

  Public from

  2024-03-25

• Supports the claim: arXiv (Bai et al., NUS Show Lab / AWS), 2024-04-29

  MLLMs often generate outputs that are inconsistent with the visual content, a challenge known as hallucination, which poses substantial obstacles to their practical deployment

  Why it matters here: A research survey posted shortly after the post describes multimodal LLM hallucination as frequent and an obstacle to deployment. This agrees with the claim, but it falls outside the target period, so it is context only.

  Where

  Abstract

  Publisher

  arXiv (Bai et al., NUS Show Lab / AWS)

  Kind of source

  Preprint, not peer reviewed

  Describes events of

  2024-04-29

  Public from

  2024-04-29