Muddles about Models
We recorded 8 claims from this post. 3 were checked against evidence. 5 can’t be checked. 5 more are held. Read the post at its source.
Claims from this post (8)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“Of course as Fei Fei Li correctly pointed out on X, it all depends on what you mean by a “model”, but here’s the crux, same as it ever was: correlations aren’t causal, semantic models.”
Our reading: Correlations found inside LLMs are not causal, semantic world models.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Finding that some stuff correlates with space or time doesn’t mean that stuff genuinely represents space or time.”
Our reading: The correlations with space and time in Gurnee and Tegmark's findings do not show that LLMs genuinely represent space or time.
No horizon
Assessment: UnresolvedSaid
Causal claim
“It’s the same problem: correlations are fragile, and you can’t count on them.”
Our reading: GPT's failure to stick to the rules of chess, even when it plays a decent game, reflects the same problem: correlations are fragile and cannot be counted on.
No horizon Evidence as of
Limitation: A failure to follow rules does not by itself reveal the model’s internal representations.
Assessment: Can’t be checkedSaid
Observation
“I would conjecture that you can’t say to ChatGPT, let’s play a game of chess and then (say) 37 moves in pause, and get it to reliably answer how many white bishops are left on the board, nor to explain what happened to some pawn that had been taken a few moves earlier.”
Our reading: Marcus conjectures that ChatGPT cannot reliably say how many white bishops remain, or what happened to a captured pawn, partway through a chess game.
No horizon
Qualifier: conjecture
Assessment: UnresolvedSaid
Conditional forecast
“Without flexible underlying models that can reliably support inferences on novel questions, we can expect more of the same of what we are seeing over and over now: more hallucinations, more reasoning that is hit or miss, and nothing like the stability we should insist on for future AI.”
Our reading: Without flexible underlying models, AI systems will keep producing hallucinations and hit-or-miss reasoning, and will lack the needed stability.
No deadline stated Evidence as of
Only if: without flexible underlying models that reliably support inference on novel questions
Assessment: UnresolvedSaid
Causal claim
“There is no internal model of what she actually did, and hence no capacity to support reliable inference, hence no guarantee that anything it writes about her will be true.”
Our reading: LinkedIn's OpenAI-powered assistant has no internal model of what Kaya Yurieff actually did, and hence no capacity to support reliable inference about her, and hence no guarantee that anything it writes about her will be true.
No horizon Evidence as of
Limitation: No evidence identifying the version, inputs, prompts or complete outputs of the assistant tested by Yurieff.
Assessment: Can’t be checkedSaid
Conditional forecast
“To encapsulate the heart of what Ernie Davis and I said at length in our 2019 book Rebooting AI, AI is unlikely to ever be trustworthy if its makers continue to skip the step of having explicit cognitive models of the world.”
Our reading: AI is unlikely ever to be trustworthy if its makers keep skipping explicit cognitive models of the world.
No horizon
Only if: if makers continue to skip explicit cognitive models of the world
Assessment: Can’t be checkedSaid
Forecast
“And here is why all this matters: safety, which is what Tegmark really cares about, is not going to emerge anytime soon, and probably ever, from hit-or-miss systems that substitute correlation for a factual, structured, interrogable representations of the world.”
Our reading: AI safety will not emerge anytime soon, and probably never, from correlation-based systems that lack structured, interrogable representations of the world.
No horizon
Qualifier: systems that substitute correlation for structured representations
Held from this post (5)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- Attribution is uncertain
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: Any military application one might imagine of the correlational approach would fail ('be toast') precisely because that approach does not yield flexible models that can reliably support inferences on novel questions.
- The two reviews read the claim differently