Hello, Multimodal Hallucinations
We recorded 10 claims from this post. 6 were checked against evidence. 4 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Observation
“ChatGPT went utterly wild when Ernie pushed on the last one, omitting some that are there and and confabulating a standalone island in the middle of the kitchen that simply isn’t there:”
Our reading: When questioned about the kitchen image, ChatGPT's vision model left out drawers that are there and made up a kitchen island that is not there.
No horizon Evidence as of
Limitation: The page reader does not reproduce the relevant image and transcript screenshots.
Assessment: SupportedSaid
Observation
“As is by now widely known, they are a hallmark of systems that generalize based on word distribution.”
Our reading: As is by now widely known, hallucinations are a hallmark of systems that generalize based on word distribution.
No horizon Evidence as of
Limitation: This establishes neither a single hallucination rate nor that all models perform equally poorly.
Assessment: Can’t be checkedSaid
Observation
“GPT-v (the visual version of GPT) can use words like refrigerator and counter, but that doesn’t mean it actually knows what these things mean..”
Our reading: GPT-4V can use words like refrigerator and counter, but that does not show it knows what they mean.
No horizon
Assessment: SupportedSaid
Observation
“As we can see, multimodal training is not magically making hallucinations go away.”
Our reading: Multimodal training is not making hallucinations go away.
No horizon Evidence as of
Limitation: The evidence does not isolate the causal effect of multimodal training or establish that it cannot reduce hallucination rates.
Assessment: Can’t be checkedSaid
Rhetoric
“The old systems were lost in a sea of correlations between words; the new system are lost in a sea of correlations between words and pixels.”
Our reading: Multimodal models, like text-only ones before them, depend on correlations (now between words and pixels) instead of understanding.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“It may get fixed, with a band-aid of some sort or another, but we are confident a zillion other pictures like it will probably yield similar errors.”
Our reading: Even if this specific kitchen example gets patched, many other similar pictures will probably still produce similar errors in multimodal LLMs.
No horizon
Qualifier: even if this specific example is patched
Assessment: UnresolvedSaid
Observation
“To our knowledge, no LLM-based system can yet reliably do what SHRLDU tried to do then:”
Our reading: As far as the authors know, no LLM-based system can yet reliably do the blocks-world modeling that SHRDLU attempted in the 1970s.
No horizon Evidence as of
Limitation: Planning benchmarks cover only part of the ordinary SHRDLU comparison.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The whole point of Rebooting AI was that the AI of 2018 didn’t actually understand how the world works, and that we couldn’t trust AI that didn’t. We stand fully by what we said.”
Our reading: The AI of 2018 did not actually understand how the world works, AI that does not understand how the world works cannot be trusted, and the authors fully stand by both points from Rebooting AI.
No horizon
Assessment: UnresolvedSaid
Forecast
“And no matter what investors and CEOs might tell you, hallucinations will remain inevitable.”
Our reading: Hallucinations in large language models will remain inevitable.
No deadline stated Evidence as of
Limitation: A mathematical reading and a practical-risk reading could yield different assessments.
Assessment: UnresolvedSaid
Causal claim
“But in other ways, large language models are a profound step backwards; they lack stable models of the world, which means that they are poor at planning, and unable to reason reliably; everything becomes hit or miss..”
Our reading: LLMs lack stable world models, which makes them poor at planning and unable to reason reliably.
No horizon Evidence as of
Limitation: Evidence supporting the behavioral component is not sufficient to score the entire causal proposition supported.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: Marcus offers to take Reid Hoffman's bet for $100,000, betting against Hoffman's claim that hallucinations can be brought down to the human-expert rate within months.