Misplaced panic over AI progress
We recorded 10 claims from this post. 10 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“Claude Code is a real advance; Mythos probably builds on some of what is learned there.”
Our reading: Claude Code is a real advance, and Mythos probably builds on some of what was learned from it.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“For exactly that reason, Wildeford’s concern about Mythos “breaking the graph” is a red herring.”
Our reading: Wildeford's concern that Mythos has 'broken' the METR time-horizon graph is a red herring, because the graph only measures 50% success.
No horizon
Qualifier: because the graph measures only 50% success
Assessment: Can’t be checkedSaid
Observation
“More broadly, the key problem with GenAI has been reliability; a graph that demands only 50% success does not address reliable performance. At all.”
Our reading: Reliability has been the key problem with GenAI, and the METR graph at 50% success does not address reliable performance at all.
No horizon
Assessment: Can’t be checkedSaid
Observation
“(Incidentally, this is yet another vindication of neurosymbolic AI – not proof that LLMs themselves can be perpetually scaled. Nor a proof that another trillion dollars will indefinitely continue the trends shown in the graph.)”
Our reading: Recent progress vindicates neurosymbolic AI and does not prove LLMs can be perpetually scaled or that another trillion dollars will continue the trend indefinitely.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“As noted in my newsletter, a lot of the advance in recent months is likely from the incorporation of symbolic tools (like code interpreters, verification, and harnesses) rather than from model scaling per se.”
Our reading: Much of the recent-months AI advance is likely due to symbolic tools like code interpreters, verification and harnesses rather than model scaling itself.
No horizon
Qualifier: recent months
Assessment: Can’t be checkedSaid
Observation
“Instead, its techniques likely work best with things like coding and math, where formal verification (good old symbolic AI for the win!) can straightforwardly apply.”
Our reading: Mythos's techniques likely work best with things like coding and math, where formal verification can straightforwardly apply.
No horizon
Scope: things like coding and math
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“To anticipate that Anthropic will have $2T revenue in 2030 is a perfect example of what I have often called the trillion pound baby fallacy”
Our reading: Anticipating that Anthropic will have $2T revenue in 2030 is a perfect example of the trillion pound baby fallacy.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“some types of challanges (e.g., reasoning accurately with respect to world models, reducing hallucinations etc) may simply not be amenable to current approaches”
Our reading: AI progress may hit limits from resource constraints, benchmark-focused tooling, formal verification limits in less formal problems, and challenges like world-model reasoning and hallucinations not amenable to current approaches.
No horizon
Assessment: Can’t be checkedSaid
Observation
“And most importantly, solving (some aspects of) software design is not open-ended intelligence. AI is definitely getting better at some things, but there is no reason to think that it is close to fully general yet.”
Our reading: AI is improving at some things but there is no reason to think it is close to fully general intelligence; software design progress is not open-ended intelligence.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In short, there is no need (yet?) to panic.”
Our reading: In short, there is no need (yet?) to panic over the AI progress shown by the METR graph.
No horizon
Qualifier: (yet?) — tentative, questioning time qualifier
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.