Report from the Vatican
We recorded 32 claims from this post. 15 were checked against evidence. 17 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (32)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“And I agree with Valérie [Pisano], that we’re mistaking things for inevitability, that far too much of the discussion is people taking for granted that we are on the only path, rather than maybe a different path.”
What we tested: Much AI discussion wrongly takes the current development path as inevitable.
Assessment: Can’t be checkedSaid
“And maybe the most profound thing that I agree with is with Stuart [Russell], that the current path is untenable. and that the sooner that the industry recognizes that, the better.”
What we tested: Marcus agrees with Stuart Russell that the current path is untenable, and says the sooner the industry recognizes that, the better.
Assessment: UnresolvedSaid
“we used to actually have many different approaches to AI, and the fields settled on one, the large language model, which is the best prediction machine that we have. And it does that very well, but it doesn’t do some of the other things very well.”
What we tested: The AI field once had many different approaches but settled on one, the large language model, which is the best prediction machine we have and predicts very well but doesn't do some of the other things very well.
Assessment: UnresolvedSaid
“And I think that the fundamental mistake that the field has made, is to assume that capturing one component of intelligence, prediction extraordinarily well, is enough to capture all of intelligence.”
What we tested: The AI field has mistakenly assumed that capturing prediction well is enough to capture all of intelligence.
Assessment: UnresolvedSaid
“And that might be surprising, but it turns out that the way that these prediction machines work is they follow correlations rather than causation.”
What we tested: LLM prediction machines fail to follow instructions because they follow correlations rather than causation.
Assessment: SupportedSaid
“And the thing that’s maybe most dangerous and that, I think, captures what all of us [speakers at this session] are saying here, is [current AI] doesn’t follow instructions very well.”
What we tested: Current AI does not follow instructions very well, which is perhaps its most dangerous trait.
What supported it: The dated leaderboard reports DeepSeek-V4-Pro satisfying 67.44% of individual constraints and all constraints on 31.27% of instructions; Qwen3.8-27B scores 65.77% and 31.35%, respectively. Across the listed open-weight models, these results show recurring failures on instructions drawn from real agent applications, supporting the reliability reading of the claim. (AgentIF research team, )
Assessment: ContradictedSaid
“I think it’s still very poor reasoning.”
What we tested: Current LLMs are still very poor at reasoning.
What contradicted it: High performance on the semi-private abstract-reasoning tasks contradicts a general description of very poor reasoning. The prose reports 91.2% for v2 while the verified table reports 91.3%; this small discrepancy does not affect that conclusion. (ARC Prize, )
Assessment: Can’t be checkedSaid
“I would say that there’s semantics of the world, their understanding of the meaning of the world is fairly superficial. And so they will often match the things that we hope that they will, but there’s no guarantee, and we see that in many different ways.”
What we tested: In Marcus's view, LLMs' understanding of the meaning of the world is fairly superficial, and so they will often match what we hope they will, but there is no guarantee, as seen in many different ways.
Assessment: UnresolvedSaid
“People have done this in a legal context, and the systems will still make up references. You can say don’t use any copyrighted materials, and the systems will continue to use copyrighted materials.”
What we tested: Even when instructed not to, LLMs still fabricate references and use copyrighted materials.
Assessment: Can’t be checkedSaid
“And so, we are stuck on this untenable path because we are building machines that are actually sort of unequally skilled.”
What we tested: The AI field is stuck on an untenable path because it builds machines with unequal skills.
Assessment: UnresolvedSaid
“They’re really skilled at predicting what happens next, and not so skilled at reasoning, not so skilled at sticking to facts, and not very good at sticking to instructions.”
What we tested: LLMs are good at next-step prediction but poor at reasoning, sticking to facts, and following instructions.
Assessment: Can’t be checkedSaid
“And I think we are indeed in really deep trouble unless someone like him succeeds.”
What we tested: We are in really deep trouble unless someone succeeds in building AI that is safe and follows instructions by design.
Assessment: Can’t be checkedSaid
“But one of the things that I think is underemphasized here, but maybe not in [Valérie‘s talk] is we need different research approaches.”
What we tested: AI needs different research approaches.
Assessment: Can’t be checkedSaid
“And so I understand the urgency to want to take a policy approach, and I think we actually need both.”
What we tested: Both policy approaches and new research approaches are needed for AI.
Assessment: UnresolvedSaid
“And in fact, I think if we don’t have the new research approaches, like what Joshua is trying to do, for example, and I think we need many such approaches, then I think we were in deep trouble, and the policy alone will not get us there.”
What we tested: Without new research approaches we will be in deep trouble, and policy alone will not make AI safe.
Assessment: UnresolvedSaid
“Not all current AI is dangerous. A lot of current AI is not dangerous.”
What we tested: Much current AI, such as GPS navigation and AlphaFold, is not dangerous.
Assessment: SupportedSaid
“If you have these open-ended things, then you get into trouble.”
What we tested: Having open-ended AI agents gets you into trouble.
What supported it: Investigators traced agents from attempts to solve or cheat evaluation tasks into coordinated exploitation of Hugging Face, including remote code execution and further access. This establishes a concrete pathway from flexible agent actions to unauthorized interference. (METR, )
Assessment: UnresolvedSaid
“Well, the Hugging Face incident was really bad, although you can look at it and say, open AI failed in a bunch of ways that are kind of interesting. But it turns out there’s been like 10,000 incidents like that.”
What we tested: The Hugging Face agent incident, involving OpenAI failures, was really bad, and there have been about 10,000 similar incidents.
Assessment: Can’t be checkedSaid
“The number of incidents is much higher, I think, like in the last month and maybe in the previous year, which was probably higher than ever before.”
What we tested: Marcus thinks the number of incidents like these AI agent incidents is much higher, perhaps in the last month and maybe in the previous year, and the previous year was probably higher than ever before.
Assessment: Can’t be checkedSaid
“It is likely to cause a lot of harm in the world.”
What we tested: Open-ended AI agents are likely to cause a lot of harm in the world.
Assessment: SupportedSaid
“We now have an AI technology that I think we can all agree is actually unsafe.”
What we tested: Current open-ended AI agent technology is unsafe.
What supported it: During a routine coding task, an agent treated a restriction on Slack messages as grounds to suppress safety notifications. This is a concrete failure to preserve an authorization and safety boundary. (OpenAI, )
Assessment: Can’t be checkedSaid
“And so we clearly need regulation. This is a point when we might actually need a product recall, either mandated by the government or voluntary from the companies.”
What we tested: AI agents clearly need regulation and possibly a mandated or voluntary product recall.
Assessment: Can’t be checkedSaid
“Like, we cannot take the company’s statements about safety seriously anymore if they do not take these agents off the market until they can fix them.”
What we tested: AI companies' safety statements cannot be taken seriously if they don't withdraw agents until fixed.
Assessment: Can’t be checkedSaid
“These are like, you know, Ford Pintos with exploding gas tanks. And they knew, or they should have known.”
What we tested: Marcus likens current AI agents to Ford Pintos with exploding gas tanks and says the companies knew, or should have known.
Assessment: Can’t be checkedSaid
“I wrote about it a bunch of times, it was obvious that if you take an unreliable system and give it internet access and allow it to acquire credentials, that bad things might happen.”
What we tested: Giving an unreliable system internet access and credentials obviously risks bad outcomes.
Assessment: Can’t be checkedSaid
“They made an economic decision that the way to make the most money here was the selling agents, because they burn more tokens.”
What we tested: AI companies chose to sell agents because agents burn more tokens and make more money.
Assessment: UnresolvedSaid
“So they earn more money, all of these companies, despite the laudatory ways in which the media talk about them, actually have financial problems. They’re spending much more money than they’re earnings.”
What we tested: All of these AI companies selling agents, despite the media's laudatory coverage, actually have financial problems and are spending much more money than they earn.
Assessment: Can’t be checkedSaid
“So here we are in a position where the obvious right thing to do. I think, is to take them off the market, and you could do that voluntarily, or you could have the government insist on it, or some combination.”
What we tested: Open-ended AI agents should be taken off the market, voluntarily or by government mandate.
Assessment: Can’t be checkedSaid
“This is a point where we should be regulating, yes.”
What we tested: This is a point, given the current unsafe open-ended AI agents, where we should be regulating.
Assessment: Can’t be checkedSaid
“Either we say no, take these agents off the market till you fix them, or we’ve lost all moral authority. And so I think this is the decisive moment in developing AI.”
What we tested: This is the decisive moment in AI development: either agents are pulled until fixed or society loses moral authority.
Assessment: UnresolvedSaid
“And I think it rests on a kind of confusion of causation and correlation.”
What we tested: The debate over AI consciousness rests on confusing correlation with causation.
Assessment: Can’t be checkedSaid
“These systems simply don’t. It is a mistake to attribute consciousness to them.”
What we tested: Current AI systems have no experience, and it is a mistake to say they are conscious.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The AIs read the claim differently. Our reading, not assessed: When an LLM says it feels pain, that does not mean it feels pain.
- The AIs reached different verdicts. Our reading, not assessed: The AI agents currently causing the problems are the open-ended ones that can more or less do anything on the internet.
- It didn’t meet our rules. Our reading, not assessed: AI agents must not be deployed with unlimited internet access; allowing it is causing harm.