Hard-forked! Casey Newton’s distorted portrait of Gary Marcus and AI skepticism
We recorded 11 claims from this post. 11 can’t be checked. 12 more are held. Read the post at its source.
Claims from this post (11)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We ought to prepare – now – for AI to get a lot more dangerous than it already is.”
Our reading: Society should prepare now for AI becoming much more dangerous than it already is.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If there is even a 5% chance I am wrong about diminishing returns, we still ought to be moving immediately to prepare.”
Our reading: Even a 5% chance that LLM returns are not diminishing means society should move immediately to prepare for advanced AI.
No horizon
Qualifier: if there is even a 5% chance he is wrong about diminishing returns
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We need, for example, a global treaty, and those tend to take a decade. Let’s get moving, people.”
Our reading: A global treaty is needed, as one example of preparation for more dangerous AI, and work on it should start now because such treaties tend to take a decade.
No horizon
Qualifier: for example
Assessment: Can’t be checkedSaid
Conditional forecast
“If AGI dropped tomorrow, given the almost total lack of regulation with real teeth, I seriously doubt we as a society would be ready for it.”
Our reading: If AGI arrived immediately, society would very likely not be ready for it, given the near-absence of regulation with real teeth.
No horizon
Only if: if AGI arrived tomorrow
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“As AI comes to have more power in society, AI skepticism becomes an increasingly important counterweight, and we should all want AI skepticism to be done right.”
Our reading: As AI gains power, AI skepticism becomes an increasingly important counterweight and should be done well.
No horizon
Qualifier: as AI gains more power in society
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“AI needs substantive regulation.”
Our reading: AI needs substantive regulation.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Likewise, an AI with a known tendency to hallucinate things can be used to target weapon (this as may happen soon, given the recent announcement that OpenAI will collaborate with Anduril).”
Our reading: Hallucination-prone AI may soon be used to target weapons, given OpenAI's collaboration with Anduril.
No horizon
Qualifier: given OpenAI's collaboration with Anduril
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I also always say we need hybrid systems that include LLMs as one tool among many, never that that LLMs will altogether disappear, even if they become a commodity and valuations fall.”
Our reading: AI needs hybrid systems that include LLMs as one tool among many.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“LLM’s in particular are wildly overrated.”
Our reading: LLMs in particular are wildly overrated.
No horizon
Scope: LLMs in particular, as distinct from AI generally
Assessment: Can’t be checkedSaid
Forecast
“Many downsides are already heere, and potential misuse in the military seems imminent.”
Our reading: Military misuse of LLMs seems imminent, adding to downsides such as covert racism, deepfakes and discrimination that already exist.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Newton is of course right that we need to prepare for AGI, whenever it comes.”
Our reading: Society needs to prepare for AGI, whenever it arrives.
No horizon
Qualifier: whenever AGI comes
Held from this post (12)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: LLMs still have trouble with causality, physical, psychological and mathematical reasoning, even if they now pass the specific published examples.
- The criteria are ambiguous. Our reading, not assessed: LLMs do not come close to delivering what was initially promised.
- The two reviews reached different assessments. Our reading, not assessed: Excessive enthusiasm for LLMs has drawn attention and resources away from other parts of AI.
- The criteria are ambiguous. Our reading, not assessed: AGI will eventually arrive but not soon, and will require new ideas because the field's current hypothesis has basically been disproven.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: An AI being stupid or overrated does not stop it being dangerous; such AI can be misused for propaganda and deepfakes.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Recent statements from industry leaders like Andreessen, Nadella and Pichai confirm Marcus's earlier predictions about LLM limitations.
- The record does not meet the current rules. Our reading, not assessed: Many students, perhaps most of OpenAI's users, use LLMs daily to write term papers, and coders use them heavily too.
- The two reviews read the claim differently
- The two reviews read the claim differently