𝐩(𝐝𝐨𝐨𝐦)
We recorded 24 claims from this post. 6 were checked against evidence. 18 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (24)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“Personally, despite the name of this essay, I am not all that worried about humans going extinct anytime very soon.”
Our reading: Marcus says he is not all that worried about humans going extinct anytime very soon.
No horizon
Scope: anytime very soon
Assessment: Can’t be checkedSaid
Observation
“There are in fact already some scenarios that are faintly imaginable now (e.g. bad actors using machines to create novel toxins, in ways that go awry), and over time could be more.”
Our reading: Some faintly imaginable scenarios already exist in which machines could kill everyone, such as bad actors using machines to create novel toxins, and more such scenarios could appear over time.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Perhaps extinction is vanishingly unlikely, but what I will call 𝐩(𝐜𝐚𝐭𝐚𝐬𝐭𝐫𝐨𝐩𝐡𝐞) – the chance that an incident that kills (say) one percent or more of the population, seems a lot higher.”
Our reading: Perhaps extinction is vanishingly unlikely, but the chance of an incident that kills, say, one percent or more of the population seems a lot higher.
No horizon
Qualifier: perhaps
Assessment: UnresolvedSaid
Observation
“Yet a really large fraction of the AI and AI-adjacent community doesn’t really want to talk about any of this at all.”
Our reading: A large fraction of the AI and AI-adjacent community does not want to discuss AI doom or catastrophe risk at all.
No horizon Evidence as of
Limitation: The packet contains no representative measurement of willingness to discuss these risks during the target period.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Unfortunately, the fact that we can’t predict the course of technology with any precision even a decade out doesn’t magically immunize us from its effects, nor relieve us of a moral responsbility to try to take precautions.”
Our reading: Not being able to predict technology precisely does not excuse us from the moral responsibility to take precautions against its risks.
No horizon
Assessment: Can’t be checkedSaid
Observation
“LeCun’s conclusion —that humans will always set the agenda—might be true, but he hasn’t given a real argument.”
Our reading: LeCun has given no real argument that humans will always set the agenda over AI, though that conclusion might be true.
No horizon Evidence as of
Limitation: This does not endorse LeCun’s conclusion or establish that his argument is sound.
Assessment: SupportedSaid
Observation
“When I think about how poor his company (Meta) is even at making LLMs that don’t hallucinate, I get worried.”
Our reading: Meta is poor at making LLMs that do not hallucinate.
No horizon Evidence as of
Limitation: TruthfulQA measures a challenging question set, not typical user traffic.
Assessment: Can’t be checkedSaid
Observation
“None of this means of course that I suddenly think that we are close to super-smart machines. (Spoiler alert: we are not).”
Our reading: We are not close to super-smart machines.
No horizon Evidence as of
Limitation: This is not a finding that superintelligence was near or far in 2023.
Assessment: UnresolvedSaid
Observation
“But if and when we get there, will they be dominating? Non-dominating? The honest answer is we haven’t a clue.”
Our reading: Marcus says that, honestly, we haven't a clue whether super-smart machines, if and when we get there, will be dominating or non-dominating.
No horizon Evidence as of
Limitation: The search cannot establish a universal absence of applicable research.
Assessment: Can’t be checkedSaid
Causal claim
“As Englander well knows, we will never get perfect estimates, but even imperfect estimates might help guide policy decisions as we try to balance attention to short-term and long-term risks in AI.”
Our reading: We will never get perfect estimates of p(doom) or p(catastrophe), but even imperfect estimates might help guide policy decisions as we try to balance attention to short-term and long-term AI risks.
No horizon
Qualifier: even imperfect estimates
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It’s therefore worth getting a rough idea of the number, even if we can’t achieve consensus on the exact number.”
Our reading: It is worth getting a rough estimate of p(doom) even if there is no consensus on the exact number.
No horizon
Qualifier: even without consensus on the exact number
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But imprecision is not an excuse for inaction.”
Our reading: Imprecise risk estimates do not justify failing to act on AI risk.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Unlike Yudkowsky I don’t think the probabilities are high enough—or imminent enough- to warrant bombing data centers any time soon. (Or probably ever.)”
Our reading: AI doom probabilities are not high or imminent enough to justify bombing data centers any time soon, and probably never will be.
No horizon
Assessment: UnresolvedSaid
Observation
“Some of the specific arguments Yudkowsky and Bostrom have made may well be weak, but there is no solid argument that the number is zero, or even that the risk is infinitesimal.”
Our reading: There is no solid argument that AI doom risk is zero or even infinitesimal, though some arguments by Yudkowsky and Bostrom may be weak.
No horizon Evidence as of
Limitation: A broad negative claim about all arguments cannot be established by sampling a few positions.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We waited too long on climate change; to the extent that there may be any real risk, either of outright doom or “mere” catastrophe, let’s not wait too long on AI.”
Our reading: To the extent AI carries any real risk of doom or catastrophe, society should not wait too long to address it, unlike with climate change.
No horizon
Qualifier: to the extent there may be real risk of doom or catastrophe
Assessment: Can’t be checkedSaid
Observation
“I am, at least for now, much more worried about the overapplication of mediocre AI (eg what happens if we put large language models into running military strategy or into hiring decisions that affect millions of people) than the use of AI that might be so smart it could catch its own errors.”
Our reading: Marcus says that, at least for now, he is much more worried about the overapplication of mediocre AI (e.g., putting large language models into running military strategy or into hiring decisions affecting millions) than about the use of AI that might be so smart it could catch its own errors.
No horizon
Qualifier: at least for now
Assessment: Can’t be checkedSaid
Observation
“But bluntly, uncontrollable superintelligence is by no means the only risk from AI, and from a policy perspective may not be the most important risk, given both the lowish (?) probability and the time horizon.”
Our reading: Uncontrollable superintelligence is by no means the only risk from AI and, from a policy perspective, may not be the most important risk, given both its lowish (?) probability and the time horizon.
No horizon
Qualifier: from a policy perspective
Assessment: Can’t be checkedSaid
Forecast
“I do think there is a lot of risk that AI will cause chaos.”
Our reading: There is a substantial risk that AI will cause chaos.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“no matter what you feel about that p(doom | superintelligence), whether you see it as sci-fi or not, you should be worried about p(catastrophe | widespread AI adoption)”
Our reading: Whatever one thinks about doom from superintelligence, people should worry about the probability of catastrophe given widespread AI adoption.
No horizon
Qualifier: regardless of views on superintelligence doom
Assessment: Can’t be checkedSaid
Forecast
“The risks of current AI (bias, defamation, cybercrime, wholesale disinformation, etc) are already starting to be well documented and may themselves quickly escalate, and could lead to geopolitical instability even over the next few years.”
Our reading: Documented risks of current AI such as bias, defamation, cybercrime and disinformation may escalate quickly and could cause geopolitical instability within the next few years.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“And what really gets the goat of people like Timnit Gebru and Abeba Birhane, not unreasonably, is that there is a good chance that any immediate effects will disproprotionately affect the dispossessed; they are very likely right about that.”
Our reading: Marcus says Gebru and Birhane are very likely right that there is a good chance any immediate effects of AI will disproportionately affect the dispossessed.
No horizon
Qualifier: a good chance
Assessment: Can’t be checkedSaid
Forecast
“It’s a story for another day, but clearly you can have catastrophe without mass death, and we are heading in that direction.”
Our reading: AI can bring catastrophe without mass death, and society is heading in that direction.
No horizon
Assessment: UnresolvedSaid
Forecast
“We aren’t going to be able calculate p(doom) to three decimal places.”
Our reading: No one will be able to calculate p(doom) precisely, to three decimal places.
No deadline stated Evidence as of
Limitation: Search coverage is not proof that a qualifying method does not exist.
Assessment: SupportedSaid
Observation
“this too seems like a rather large assumption that is far from proven, a possibility but not demonstrably a default,”
Our reading: Yudkowsky's claim that AGI will kill everyone by default is far from proven; it is possible but not shown to be the default.
No horizon Evidence as of
Limitation: The conclusion involves evaluating the sufficiency of arguments, not checking a numerical measurement.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Much of the risk from AI does not come from uncontrollable superintelligence.
- Attribution is uncertain