Why my p(doom) has risen, dramatically
We recorded 19 claims from this post. 3 were checked against evidence. 16 can’t be checked. 7 more are held. Read the post at its source.
Claims from this post (19)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“Part of my reasoning then was that actual malice on the part of AI was unlikely, at least any time soon.”
Our reading: As of May 2025, Marcus believed actual malice on the part of AI was unlikely any time soon.
No horizon
Qualifier: his reasoning as of May 2025
Assessment: Can’t be checkedSaid
Forecast
“But even so, I think most scenarios (e.g., people homebrewing biological weapons) could eventually be stopped, perhaps causing a lot of damage but coming nowhere near to literally extinguishing humanity.”
Our reading: Most AI misuse scenarios by bad actors could eventually be stopped, perhaps causing much damage but not extinguishing humanity.
No horizon
Qualifier: most scenarios
Assessment: Can’t be checkedSaid
Observation
“What crystallized for me over the last few days is that we have such a person.”
Our reading: Marcus says it became clear to him over the last few days that Elon Musk is a person who matches his list of things one might need to really screw up the planet.
No horizon
Qualifier: you might need something like the following
Assessment: Can’t be checkedSaid
Observation
“The fact that xAI can’t get its house together in even basic ways is disturbing.”
Our reading: xAI cannot get its house in order even in basic ways.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Worse, it is clear, even from Elon’s own posts on X, that the company is in over its head.”
Our reading: xAI is in over its head.
No horizon
Assessment: UnresolvedSaid
Observation
“The system is so bound to Musk it even asked his opinion on pizza.”
Our reading: Grok is so bound to Musk that it sometimes searches for Musk's opinion before answering, and in one case even asked his opinion on pizza.
No horizon Evidence as of
Limitation: The available demonstration establishes occurrence, not how often the behavior happened or its underlying cause.
Assessment: UnresolvedSaid
Observation
“From this I infer that xAI’s main methodology for handling alignment is trial-and-error.”
Our reading: From xAI's account of the incident, Marcus infers that xAI's main methodology for handling alignment is trial-and-error.
No horizon Evidence as of
Limitation: Public disclosures may omit important internal practices.
Assessment: Can’t be checkedSaid
Conditional forecast
“Absent systems cognitively rich enough to represent and reason about moral principles, I don’t how this can ever work.”
Our reading: Marcus does not see how building alignment through training data alone can ever work unless systems are cognitively rich enough to represent and reason about moral principles.
No horizon
Only if: absent systems cognitively rich enough to represent and reason about moral principles
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Way out of line is right.”
Our reading: Marcus agrees with Samuel Marks's thread that xAI's conduct in releasing Grok 4 without following the industry's safety conventions was way out of line.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“And industry-standard measures are unlikely to be enough, in any event.”
Our reading: Industry-standard AI safety measures are unlikely to be sufficient.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We need serious regulation to prevent AI’s from running amok. Voluntary agreements between companies are not going to cut it.”
Our reading: Serious AI regulation is needed because voluntary company agreements will not suffice.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We need liability, auditing, standards of malpractice, international treaties, too.”
Our reading: AI requires liability, auditing, malpractice standards, and international treaties.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It is genuinely scary that Elon Musk went from being one of the first industry leaders warning about AI risks to being the most reckless of the AI leaders.”
Our reading: It is genuinely scary that Elon Musk went from being one of the first industry leaders to warn about AI risks to being the most reckless of the AI leaders.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Drones and even nuclear weapons may eventually be under LLM command.”
Our reading: Drones and even nuclear weapons may eventually be placed under LLM command.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Forcing unreliable AI everywhere in the decision-making chain is not necessarily something we should want.”
Our reading: We should be wary of forcing unreliable AI throughout decision-making chains.
No horizon
Assessment: UnresolvedSaid
Observation
“My p(doom) is still lower than most industry people’s.”
Our reading: Marcus's p(doom) is lower than that of most AI industry people.
No horizon Evidence as of
Limitation: A researcher sample is not automatically representative of the AI industry.
Assessment: Can’t be checkedSaid
Forecast
“I am at maybe 3% now, much higher than a month ago, contemplating what a wealthy, reckless megalomaniac might in the worst circumstances do, But I am still betting on Team Human for the foreseeable future.”
Our reading: Marcus puts the probability of AI-caused doom at about 3%, up sharply from a month earlier, while still betting humans prevail for the foreseeable future.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Maybe someone will come to their senses and stop putting LLMs into military systems.”
Our reading: LLMs should not be put into military systems.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“ButI have seen enough to realize that there is a real risk, especially in the current anti-regulatory regime, that some exceptionally powerful person, Elon or otherwise, unconstrained in conventional ways, with a reckless disregard for humanity, could accidentally launch and spread an AI that deliberately or otherwise causes “significant harm to the world”.”
Our reading: There is a real risk, especially under the current anti-regulatory regime, that some exceptionally powerful person (Elon or someone else), unconstrained in conventional ways and recklessly disregarding humanity, could accidentally launch and spread an AI that, deliberately or otherwise, causes 'significant harm to the world'.
No horizon
Qualifier: especially in the current anti-regulatory regime
Held from this post (7)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: xAI apparently skipped or poorly performed standard preflight safety checks like red-teaming and model cards before releasing Grok 4.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: From xAI's own account of the 'mechahitler' incident, it sounds to Marcus like a handful of tiny changes (three in this case) can have huge, unpredicted consequences through Grok's system.
- The two reviews reached different assessments. Our reading, not assessed: Musk's political capital is rapidly diminishing compared with a few months ago.