Seven lies in four sentences
We recorded 6 claims from this post. 3 were checked against evidence. 3 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (6)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“Rumors that LLMs have solved then hallucination problem are greatly exaggerated.”
Our reading: LLMs have not solved the hallucination problem.
No horizon Evidence as of
Limitation: The research was a preprint at publication.
Assessment: UnresolvedSaid
Observation
“By my count, though I acknowledge I may have missed one, there are at least 7 falsehoods.”
Our reading: A recent Microsoft Bing answer about the House Speaker contained at least seven falsehoods.
No horizon Evidence as of
Limitation: The underlying political facts are more accessible than the original model output.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The worst part is not that every single sentence contains at least one lie, but that the whole thing sounds detailed and real. To someone who wasn’t following matters closely, it might well all sound plausible.”
Our reading: The worst part of Bing's false answer is not that every sentence contains at least one falsehood but that the whole thing sounds detailed and real, so that to someone who wasn't following matters closely it might well all sound plausible.
No horizon
Scope: to someone who wasn't following matters closely
Assessment: Can’t be checkedSaid
Rhetoric
“Ezra and I harped on their propensity to generate bullshit because bullshit was and is their fundamental product.”
Our reading: Bullshit was and still is the fundamental product of LLMs.
No horizon
Assessment: UnresolvedSaid
Causal claim
“This all happens not because LLMs are smart, in the sense of being able to compare what they say with the world, but simply because they are good at making new sentences that sound old sentences, the essence of autocomplete.”
Our reading: LLMs produce plausible falsehoods because they mimic existing sentences like autocomplete rather than checking what they say against the world.
No horizon Evidence as of
Limitation: Training descriptions establish design facts, not a complete causal explanation of errors.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“we really ought to commit to building AI that can.”
Our reading: The field should commit to building AI that can distinguish facts from fiction.
No horizon
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.