This Week in AI Doublespeak
We recorded 3 claims from this post. 2 were checked against evidence. 1 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (3)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Observation
“if the definition of reasoning is to obtain valid conclusions from known facts, GPT-4 frequently falls short there, too.”
Our reading: GPT-4 frequently fails to reason, if reasoning means reaching valid conclusions from known facts.
No horizon Evidence as of
Limitation: Later evaluations were not rejected merely because they were published after the post; the problem is their uncertain connection to the target model period.
Assessment: SupportedSaid
Observation
“True LLMs don’t (just) memorize, and true that their models make for lousy databases (for a database a hallucination is an outright fail)”
Our reading: OpenAI's LLMs make lousy databases because hallucinations amount to outright failures for a database.
No horizon Evidence as of
Limitation: This does not establish that most answers were false.
Assessment: Can’t be checkedSaid
Observation
“So-called “reasoning” by free association, even constrained by a giant database, isn’t really reasoning:”
Our reading: The free-association 'reasoning' LLMs do is not really reasoning, even when constrained by a giant database.
No horizon
Qualifier: even when constrained by a giant database
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.