Claims with a mixed outcome
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims with a mixed outcome (32)
Part of the claim holds and part does not, and the record says which part is which.
Showing 32 of 32 claims, ordered by the date first said, newest first.
Assessment: MixedSaid
Gary Marcus Observation
“From the initial report, we can tell almost nothing.”
Our reading: From OpenAI's initial report, we can tell almost nothing about the new system.
No horizon Evidence as of
Limitation: The outcome depends on reading the sentence broadly in light of the preceding methodological criticisms. A reading confined to generality beyond mathematics would differ.
Assessment: MixedSaid
Gary Marcus Observation
“SpaceX, increasingly pitched as an AI play, has lost steam (up slightly for the day, but down 11.74% for the week), and its bonds are doing badly, too:”
Our reading: SpaceX, increasingly pitched as an AI play, has lost steam, being up slightly for the day but down 11.74% for the week, and its bonds are doing badly too.
No horizon Evidence as of
Limitation: The exact intraday snapshot and chart baseline are missing.
Assessment: MixedSaid
Gary Marcus Observation
“They had a lead but they squandered it.”
Our reading: OpenAI had a lead in AI but squandered it.
No horizon Evidence as of
Limitation: Different evaluations measure different capabilities and preferences.
Assessment: MixedSaid
Ed Zitron Causal claim
“This issue bubbled to the surface in the last few months because Anthropic and OpenAI both quietly moved all of their enterprise customers to token-based billing in Q1 2026, and because these enterprise customers are run by Business Idiots with no connection to actual work, CEOs encouraged (or actively incentivized) their workers to use AI as much as possible, in some cases even making one’s AI use a KPI that could cost them their job.”
Our reading: The AI cost problem surfaced in the last few months because Anthropic and OpenAI quietly moved all their enterprise customers to token-based billing in Q1 2026, and because CEOs had encouraged or incentivized workers to use AI as much as possible, in some cases making AI use a job-threatening KPI.
No horizon Evidence as of
Limitation: Anthropic's migration record does not establish OpenAI's Q1 customer coverage [e1].
Source: AI Doesn't Have ROI
Assessment: MixedSaid
Gary Marcus Observation
“And the differences are immense; one (the LLM) effectively memorizes the entire internet; the other (the human) builds a mental model through experience with world.”
Our reading: LLMs differ immensely from humans: LLMs effectively memorize the internet, while humans build mental models through experience of the world.
No horizon Evidence as of
Limitation: The result depends on reading the sentence as a contrast between modes of learning, rather than merely saying humans and LLMs differ.
Source: The Pope appears to understand AI better than Geoffrey Hinton does.
Assessment: MixedSaid
Gary Marcus Observation
“Hassabis is no longer on board either, and nor are Sutskever and LeCun. Nadella and Pichai have also hinted at skepticism around scaling.”
Our reading: Hassabis, Sutskever and LeCun no longer believe in scaling as the route to AGI, and Nadella and Pichai have hinted at skepticism about it.
No horizon Evidence as of
Limitation: The record does not establish earlier endorsement and subsequent rejection of scaling alone for everyone described as having changed position.
Source: BREAKING: Sam Altman concedes that we need major breakthroughs beyond mere scaling to get to AGI
Assessment: MixedSaid
Ed Zitron Causal claim
“Deep Impact, my analysis of the DeepSeek situation and why it scared the American AI industry (clue: it's cost-related and nothing to do with "national security").”
Our reading: DeepSeek scared the American AI industry because of cost, not because of national security.
No horizon Evidence as of
Limitation: The market report does not establish individual investors' or executives' private motives [e1].
Source: 2025, A Retrospective
Assessment: MixedSaid
Gary Marcus Observation
“Coke and Pepsi are commodities; genAI models are, too.”
Our reading: Generative AI models are commodities.
No horizon Evidence as of
Limitation: The buyer evidence comes from a single survey of builders by a venture firm, from mid-2025, months before the December 2025 post.
Source: The core misconception that is driving American AI policy
Assessment: MixedSaid
Gary Marcus Observation
“productivity studies sometimes show a 30% gain, but none have come close to 10x (1000% gain).”
Our reading: No productivity studies of LLMs have shown anything close to a 10x gain; some show around 30%.
No horizon Evidence as of
Limitation: The counterexample involved six fictional cases and document production, not overall physician productivity.
Source: Three years on, ChatGPT still isn't what it was cracked up to be – and it probably never will be
Assessment: MixedSaid
Gary Marcus Observation
“What Altman couldn’t say then was that he has a plan, to reduce the cost of his borrowing … by having the American taxpayer (indirectly) foot the bill.”
Our reading: OpenAI plans to reduce its borrowing costs by having American taxpayers indirectly bear the cost.
No horizon Evidence as of
Limitation: The original remarks and clarification are reproduced by news outlets; a complete primary transcript was not obtained.
Source: OpenAI probably can’t make ends meet. That’s where you come in.
Assessment: MixedSaid
Gary Marcus Observation
“Some customers even wanted the old models back. Almost nobody felt satisfied.”
Our reading: Almost no one was satisfied with GPT-5, and some customers wanted the old models back.
No horizon Evidence as of
Limitation: No representative satisfaction survey of GPT-5 users from the launch week was found, so the share of satisfied users is not known.
Assessment: MixedSaid
Gary Marcus Observation
“Cherry-picked studies, benchmark-gaming, and now even vibe-coded graphs, with zero transparency about how systems work or how they have been trained; public science is in the rear view mirror.”
Our reading: AI has been marked by cherry-picked studies, benchmark-gaming and now even vibe-coded graphs, with zero transparency about how systems work or how they have been trained, and public science is in the rear view mirror.
No horizon Evidence as of
Limitation: The judgment depends on reading zero transparency as no substantive disclosure, rather than insufficient disclosure for reproduction.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: MixedSaid
Gary Marcus Observation
“Congress has failed to pass any meaningful AI regulation, and even worse, they are now actively aiming to prevent States — probably our last hope — from passing anything meaningful.”
Our reading: Congress has passed no meaningful AI regulation and is trying to prevent states from passing meaningful AI regulation.
No horizon Evidence as of
Limitation: Congressional passage does not establish enactment or that the bill's duties were already in force.
Source: Two Years Ago Today in AI History: The Tale of An About-face in AI Regulation
Assessment: MixedSaid
Gary Marcus Observation
“Government officials aren’t worrying out loud about the risks of AI, anymore. They are downplaying them.”
Our reading: US government officials now downplay rather than voice concern about AI risks.
No horizon Evidence as of
Limitation: The Federal Reserve is part of the federal government but does not speak for the administration.
Source: Two Years Ago Today in AI History: The Tale of An About-face in AI Regulation
Assessment: MixedSaid
Gary Marcus Observation
“METR’s blog and its tweets (perhaps written by publicists or by generative AI rather than by scientists), however, dropped the careful qualifications and nuance of the technical paper and made claims that go far beyond anything that the study actually supports.”
Our reading: METR's blog and tweets made claims far beyond what its study supports, dropping the technical paper's qualifications.
No horizon Evidence as of
Limitation: This assesses the identified announcements, not every METR tweet.
Source: The latest AI scaling graph - and why it hardly makes sense
Assessment: MixedSaid
Gary Marcus Causal claim
“As a fantastic, souped-up version of autocomplete, AI coding tools somewhat increase productivity.”
Our reading: AI coding tools, acting as a fantastic, souped-up version of autocomplete, somewhat increase productivity.
No horizon Evidence as of
Limitation: Neither study establishes an industry-wide average effect or isolates autocomplete from other assistance features [e1, e2].
Source: AI Coding Fantasy meets Pac-Man
Assessment: MixedSaid
Gary Marcus Observation
“Critical things like factuality, compositionality, and common sense aren’t even addressed.”
Our reading: The ARC test does not address factuality, compositionality, or common sense.
No horizon Evidence as of
Limitation: The result depends on how 'common sense' and 'compositionality' are defined. A narrower reading, such as broad everyday world knowledge, would favor Marcus.
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: MixedSaid
Gary Marcus Observation
“In January, in IEEE Spectrum, Reid Southen and I demonstrated similar issues in video generation software, such as Midjourney and Dall-E:”
Our reading: In January, in IEEE Spectrum, Marcus and Reid Southen demonstrated issues similar to the New York Times findings in what he called video generation software, such as Midjourney and Dall-E.
No horizon Evidence as of
Limitation: The mixed result turns on retaining the explicit word 'video'.
Source: Generative AI’s Continuing Copyright Problems, an Essay in Memory of Suchir Balaji, 1998 - 2024
Assessment: MixedSaid
Gary Marcus Observation
“I strongly believe the current results are robust.”
Our reading: Marcus strongly believes that the current results on LLMs' reasoning failures, including the Apple study and the other work he cites, are robust.
No horizon Evidence as of
Limitation: 'I strongly believe' is a statement of belief; only the robustness of the results is graded.
Source: LLMs don’t do formal reasoning - and that is a HUGE problem
Assessment: MixedSaid
Gary Marcus Observation
“The graph assumes that all continued progress will be same — ignoring qualitatively unsolved problems (hallucinations, planning, reasoning etc), the fact that synthetic data may not be as useful as natural data, limits on then-possible energy distribution and compute, etc.”
Our reading: Aschenbrenner's graph assumes all continued progress will be the same, ignoring qualitatively unsolved problems such as hallucinations, planning and reasoning, the possibility that synthetic data may not be as useful as natural data, and limits on energy distribution and compute.
No horizon Evidence as of
Limitation: The result depends on including the graph's supporting analysis.
Source: AGI by 2027?
Assessment: MixedSaid
Gary Marcus Observation
“Instead, everything that LLMs say -- true or false -- comes from the same process of statistically reconstructing what words are likely in some context. They NEVER fact-check what they say.”
Our reading: All LLM output, true or false, comes from the same statistical word-reconstruction process, and LLMs never fact-check what they say.
No horizon Evidence as of
Limitation: The packet documents next-token generation only for GPT-4 and does not cover every LLM in use during the period.
Assessment: MixedSaid
Gary Marcus Observation
“The study Mollick linked to doesn’t actually show what he claims. If you read it carefully, it says literally nothing about capabilities improving.”
Our reading: The study Mollick cited does not show that LLM capabilities are improving; it only addresses compute efficiency.
No horizon Evidence as of
Limitation: Language-model performance is narrower than general capability.
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: MixedSaid
Gary Marcus Observation
“The company Sam and Greg built has little to do with what was originally promised.”
Our reading: The OpenAI that Altman and Brockman built has little to do with what was originally promised.
No horizon Evidence as of
Limitation: The weighting of different founding commitments is an evaluator interpretation.
Source: Elon Musk sues OpenAI
Assessment: MixedSaid
Gary Marcus Observation
“So let me get this straight? Sora’s can’t reliably handle basic physics, ChatGPT had an unexplained meltdown, Gemini can’t even remember who was on Apollo 11, there are no formal guarantees that any of this will ever work, you yourself admit you don’t know how it works, or how much better it will get next time around, and you want another $7 trillion dollars?”
Our reading: ChatGPT had an unexplained meltdown.
No horizon Evidence as of
Limitation: The incident was initially unexplained. Reading the adjective as referring only to that initial stage would change the assessment.
Source: Belief in magic may be declining
Assessment: MixedSaid
Gary Marcus Observation
“Special-purpose AI has been great for static, easily-simulated games with rules that have been stable for centuries, like Chess, Go, and even Diplomacy, but never been great at dynamic environments in which the environment itself is unpredictable.”
Our reading: Special-purpose AI has been great for static, easily simulated games with rules that have been stable for centuries, such as chess, Go and even Diplomacy, but has never been great at dynamic environments in which the environment itself is unpredictable.
No horizon Evidence as of
Limitation: The distinction between environmental uncertainty and unrestricted task change affects the result.
Assessment: MixedSaid
Gary Marcus Observation
“The New York Times versus OpenAI lawsuit made that abundantly clear, with a hundred examples like this one, in which ChatGPT reconstructed entire stretches of text (red) from brief prompts2 (black text in top line):”
Our reading: The NYT v. OpenAI lawsuit showed about a hundred examples of ChatGPT reconstructing entire stretches of text from brief prompts.
No horizon Evidence as of
Limitation: Both filings are party submissions, not judicial findings about experimental validity.
Assessment: MixedSaid
Gary Marcus Observation
“The spirit of the original arrangement was that everything that the for-profit did was supposed to be in the service of the non-profit. A board with no financial interest was supposed to look out for humanity.”
Our reading: OpenAI's original structure was meant to make the for-profit serve the non-profit, with a financially disinterested board looking out for humanity.
No horizon Evidence as of
Limitation: This assesses the intended structure, not its effectiveness.
Assessment: MixedSaid
Gary Marcus Observation
“In many cases, a naive user might take the results, invariably presented as fact, as correct, when they are not.”
Our reading: GPT-4 with plug-ins invariably presents its results as fact, and in many cases its results are incorrect even though a naive user might take them as correct.
No horizon Evidence as of
Limitation: The mixed result depends on reading invariably literally.
Source: Getting GPT to work with external tools is harder than you think
Assessment: MixedSaid
Gary Marcus Causal claim
“It makes current AI hard to debug (nobody knows for example how to ground them in facts), and it makes them difficult predict, which means, unlike calculators or spreadsheets, we don’t really know what’s going to happen when we ask them a question.”
Our reading: Because we don't really understand how trained neural networks do what they do, current AI is hard to debug (for example, nobody knows how to ground them in facts) and difficult to predict, so unlike calculators or spreadsheets we don't really know what will happen when we ask them a question.
No horizon Evidence as of
Limitation: The supplied Anthropic passage describes interpretability obstacles, rather than directly measuring difficulty diagnosing or fixing failures; its demonstrated progress concerns a small model [e1].
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: MixedSaid
Gary Marcus Observation
“Two reasons, first, because I believe that more incidents like this will happen, perhaps many more, hundreds or thousands, given how rapidly chatbots are being deployed, and the near total lack of regulation on how they can be used.”
Our reading: There is a near-total lack of regulation on how chatbots can be used, and they are being deployed rapidly.
No horizon Evidence as of
Limitation: The result depends on reading regulation to include existing binding law.
Assessment: MixedSaid
Gary Marcus Observation
“CNET became the first casualty of the recently fashionable tendency to put too much faith in ChatGPT.”
Our reading: CNET was the first casualty of putting too much faith in ChatGPT, publishing AI-written stories with many mistakes.
No horizon Evidence as of
Limitation: The ChatGPT denial comes from CNET itself, and the underlying model was not disclosed.
Assessment: MixedSaid
Gary Marcus Causal claim
“Debugging with deep learning is wicked hard, because nobody (a) really understands how it works, and (b) knows how to fix problems other to collect more data and add more layers and so forth.”
Our reading: Debugging deep learning is wicked hard because nobody really understands how it works or knows how to fix problems other than collecting more data, adding more layers and so forth.
No horizon Evidence as of
Limitation: The evidence concerns neural language models, not every deep learning application.
Source: Dear Elon Musk, here are five things you might want to consider about AGI
No claims match. Remove a filter or change the search, or clear everything.