Claims that can’t be checked, page 21 of 37
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims that can’t be checked (3643)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 21 of 37: claims 2001 to 2100 of 3643, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The entirely speculative conjecture that any profits will actually do much to help the American people, as opposed to just enriching those who own that infrastructure.”
Our reading: The idea that any AI infrastructure profits will much help the American people, rather than just enrich the owners, is entirely speculative.
No horizon
Source: Hot take on an AI catfight
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Sam Altman himself hinted we were close, just two weeks ago, writing “We are now confident we know how to build AGI as we have traditionally understood it”, which to be honest gave me Elizabeth Holmes vibes.”
Our reading: Marcus says Altman's statement that OpenAI is confident it knows how to build AGI gave him, to be honest, Elizabeth Holmes vibes.
No horizon
Qualifier: personal impression ('gave me ... vibes')
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“I should also be clear that the reason they're doing this is that they're desperate.”
Our reading: Google and Microsoft are forcing generative AI features and price increases on customers because they are desperate for growth.
No horizon
Source: The Slop Society
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“Tech has already taken liberties with the digital experience and I believe the Slop Era will be one where experiences will begin to subtly and overtly rot, with companies proudly boasting that they're making "adjustments to user engagement that will provide better business-forward outcomes," which will be code for "make them worse so that we make more money."”
Our reading: In the coming 'Slop Era', tech companies will deliberately degrade user experiences to make more money.
No horizon
Source: The Slop Society
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“I believe, however, that Zuckerberg is attempting to move it further — to publicly say "we're going to fill your feeds with AI slop" and "we're firing 5% of our workers because they suck" and "we're going to have AI profiles instead of real people on the site you visit to see real people's stuff"”
Our reading: Zitron believes Zuckerberg is attempting to move the Overton window further by publicly signalling, in Zitron's paraphrase, that Meta will fill feeds with AI slop, is firing 5% of workers because they suck, and will have AI profiles instead of real people.
No horizon
Qualifier: I believe
Source: The Slop Society
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“The Information's Jon Victor and Aaron Holmes add that it's yet to be seen what Microsoft does with the corporate customers using Microsoft's 365 productivity suite — adding Copilot costs $30-per-user-per-month — but my hypothesis is it will do exactly the same thing that Google did to its business customers.”
Our reading: Zitron hypothesizes that Microsoft will do exactly the same thing to its corporate Microsoft 365 customers that Google did to its business customers, namely raising prices along with adding AI features (Copilot) they don't want.
No horizon
Qualifier: my hypothesis
Source: The Slop Society
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“But not this year (or next, or the one after that, and probably not this decade, except in narrow use cases).”
Our reading: AI agents probably will not do a huge fraction of human knowledge work this decade, except in narrow use cases.
No horizon
Qualifier: except in narrow use cases
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“A system answering a river-crossing problem should not answer by superficial analogy to the words in similar problems found in a giant scrape of the internet, but rather by reasoning through concepts about boats, rivers, people, and so on.”
Our reading: AI systems should solve problems by reasoning over concepts, not by surface analogy to text scraped from the internet.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Anyone who tries to tell you otherwise is either confused or trying to hype something.”
Our reading: Anyone claiming AGI has been achieved is either confused or hyping something.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“but that regurgitation is an illusion. Genuine depth is lacking.”
Our reading: The depth LLMs seem to have at times, from regurgitating or replicating deep analyses found on the internet, is an illusion; genuine depth is lacking.
No horizon
Qualifier: at times (per Mahabal's quoted observation)
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Moving past broad but shallow AI and into a new and hopefully safer and more reliable regime should be a major research priority for the human species.”
Our reading: Humanity should make it a major research priority to move beyond broad, shallow AI toward safer, more reliable AI.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Counting on AI that is shallow and not reliable, it should also be said, could cause huge problems.”
Our reading: Relying on shallow, unreliable AI could cause huge problems.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Then, and only then, will we have reliable solutions to the unreliability that has plagued recent AI.”
Our reading: AI's unreliability will be reliably solved only once systems reason over concepts with deep understanding.
No horizon
Only if: only if systems achieve deep conceptual understanding
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“For all these reasons, I propose that we call LLMs broad, shallow intelligence..”
Our reading: Marcus proposes that LLMs should be called 'broad, shallow intelligence'.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Instead, LLMs are still unpredictable enough that humans should still be in the loop in most case.”
Our reading: Because LLMs remain unpredictable, humans should stay in the loop in most cases.
No horizon
Qualifier: in most cases
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“We can never guarantee that broad but shallow AI will be safe, or that it will be human-compatible (in the words of Stuart Russell), or that it will do what we ask of it.”
Our reading: Broad but shallow AI can never be guaranteed to be safe, human-compatible, or to do what we ask.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Every declaration of us already reaching AGI I have seen rests on redefining the term downwards, as I discussed last week,”
Our reading: Every claim he has seen that AGI has already been reached depends on lowering the definition of AGI.
No horizon Evidence as of
Limitation: The evidence does not show which declarations Marcus saw. It also does not show whether he saw these two.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Nobody serious thinks that we have reached AGI (artificial general intelligence) yet.”
Our reading: No serious person thinks that AGI (artificial general intelligence) has been reached yet.
No horizon
Scope: among people he counts as serious
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The river crossing example and many others shows that LLMs often use the words without a deep understanding what those words mean.”
Our reading: LLMs often use words without deeply understanding what they mean.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“o3 is a real advance in some domains beyond GPT-4, but it’s likely not the quantum leap forward people would expect of the name GPT-5, and at a rumored $1000/querry certainly not at the price point anyone might have hoped for”
Our reading: o3 is a real advance beyond GPT-4 in some domains, but it is likely not the quantum leap people would expect of the name GPT-5, and at a rumored $1000 per query it is certainly not at the price point anyone might have hoped for.
No horizon
Scope: in some domains
Source: AGI isn’t coming in 2025, and GPT-5 probably isn’t either.
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Gary Marcus wishes the media would hold those who make unrealistic promises to account.”
Our reading: The media should hold those who make unrealistic AI promises to account.
No horizon
Source: AGI isn’t coming in 2025, and GPT-5 probably isn’t either.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“We might see GPT-5 this year, but given bottlenecks on data and inherent limits in LLMs we may well not.”
Our reading: GPT-5 may well not be released in 2025, owing to data bottlenecks and inherent limits of LLMs.
No horizon
Qualifier: given bottlenecks on data and inherent limits in LLMs
Source: AGI isn’t coming in 2025, and GPT-5 probably isn’t either.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“He made some of his usual overpromises (500,000 humanoid robots in three years, which seems to very unlikely)”
Our reading: Musk's promise of 500,000 humanoid robots in three years seems very unlikely to be fulfilled.
No horizon
Qualifier: seems
Source: AGI isn’t coming in 2025, and GPT-5 probably isn’t either.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I won’t be at all surprised if don’t even see anything worthy of the GPT-5 name”
Our reading: It is quite possible that no model worthy of the GPT-5 name will appear in 2025.
No horizon
Source: AGI isn’t coming in 2025, and GPT-5 probably isn’t either.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“There are some applications where the current systems are reliable enough to be practically useful, though not nearly as many as is often claimed.”
Our reading: Current AI systems are reliable enough to be useful in some applications, but far fewer than often claimed.
No horizon Evidence as of
Limitation: The claim identifies no advertised comparison set, common unit for counting applications, or standard for practical reliability.
Source: The Five Stages of AGI Grief
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If a lot of people’s goalposts have moved, mine have not. same as before, I will be satisfied if and when we reach AI systems are “flexible and general, with resourcefulness and reliability comparable to (or beyond) human intelligence” – and not until.”
Our reading: Marcus's goalposts for AGI have not moved: he will be satisfied if and when AI systems are flexible and general, with resourcefulness and reliability comparable to or beyond human intelligence, and not until.
No horizon
Qualifier: if and when AI systems are flexible and general, with resourcefulness and reliability comparable to (or beyond) human intelligence
Source: The Five Stages of AGI Grief
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“In reality, the definition of AGI, while not perfect, is reasonably clear — and has been reasonably clear — for a long time.”
Our reading: The definition of AGI, though imperfect, is reasonably clear and has been for a long time.
No horizon
Source: The Five Stages of AGI Grief
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Current models trained on next-token prediction have remarkable abilities and remarkable weaknesses. The scope of those abilities and weaknesses of the current systems is not well understood.”
Our reading: Current next-token prediction models have remarkable abilities and weaknesses, whose scope is not well understood.
No horizon
Source: The Five Stages of AGI Grief
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“But so far I have not seen a principled solution to even one of these longstanding problems, and that tells me a lot.”
Our reading: So far Marcus has not seen a principled solution to even one of these longstanding problems, and he says that tells him a lot.
No horizon
Qualifier: so far
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“[Note also that the current idea, test-time “scaling”, is very expensive, and will likely prove unreliable out of domains where augmentation is adequate, in which case it will get comparatively little commercial traction.”
Our reading: Test-time scaling is very expensive and will likely prove unreliable outside domains where data augmentation is adequate.
No horizon
Qualifier: outside domains where augmentation is adequate
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“In domains where that’s not possible, o1 may not even be that much better than GPT-4 (e.g on some language tasks), as you can see if you read OpenAI’s reports carefully.”
Our reading: Where data augmentation isn't possible, o1 may not be much better than GPT-4, for example on some language tasks.
No horizon
Qualifier: in domains where data augmentation isn't possible
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“o1 probably isn’t as robust as people might have hoped. Even on certain benchmarks, results may be fragile or difficult to replicate:”
Our reading: o1 is probably less robust than hoped, and some benchmark results may be fragile or hard to replicate.
No horizon
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“In o1’s best performances cases, there may have been heavy data augmentation that is not possible in more open-ended domains.”
Our reading: In o1's best-performance cases there may have been heavy data augmentation that is not possible in more open-ended domains.
No horizon
Scope: in o1's best-performance cases
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Here is a thread of links to a several recent observations that I think show how far we are from robust intelligence.”
Our reading: Marcus thinks several recent observations show how far AI is from robust intelligence.
No horizon
Qualifier: I think
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I do not share his confidence.”
Our reading: Marcus does not share Sam Altman's confidence that OpenAI knows how to build AGI.
No horizon
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“The presentation of o3 was problematic, and no clean test of out-of-domain generalization was provided, suggesting that we may see the kinds of problems with distribution shift as ever, outside of semi-closed domains where synthetic data can readily be generated.”
Our reading: Marcus says the presentation of o3 was problematic and gave no clean test of out-of-domain generalization, which suggests o3 may show the same distribution-shift problems as ever outside semi-closed domains where synthetic data can readily be generated.
No horizon
Scope: outside semi-closed domains where synthetic data can readily be generated
Source: Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“But fundamentally its videos are more like visual collages than the renderings of game engines, which is why they sometimes have a surreal or magic trick quality.”
Our reading: Sora's videos are fundamentally more like visual collages than game-engine renderings, which is why they sometimes have a surreal or magic-trick quality.
No horizon
Qualifier: more like (comparative, not categorical)
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“In our view, it is only once AI researchers grapple directly with the challenging problems inherent in commonsense reasoning about entities that interact in space and persist in time that the field of general-purpose AI will finally begin to mature.”
Our reading: General-purpose AI will begin to mature only once researchers directly tackle commonsense reasoning about entities interacting in space and persisting in time.
No horizon
Only if: only once researchers grapple directly with commonsense reasoning about entities in space and time
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“My essay was about the small but very real possibility that 2025 could see the worst cyberattack in history, perhaps catalyzed in some fashion by generative AI (which was #24 in my list of 25 AI predictions for 2025).”
Our reading: There is a small but real possibility that 2025 sees the worst cyberattack in history, possibly catalyzed in some way by generative AI.
No horizon
Qualifier: small but very real possibility
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Technical moat will continue to be elusive. Instead, there will be more convergence on broadly similar models, across both US and China; some systems in Europe will catch up to roughly the same place.”
Our reading: Filed under 'Medium confidence': through the end of 2025, a technical moat will continue to be elusive; instead there will be more convergence on broadly similar models across both the US and China, and some systems in Europe will catch up to roughly the same place.
Deadline passed Evidence as of
Limitation: This record does not show what happened in 2025. No evidence about the gap between leading models is part of this record.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Few companies (and even fewer consumers) will adopt o3 at wide scale because of concerns about price and robustness relative to that price.”
Our reading: Filed under 'Medium confidence': by the end of 2025, few companies (and even fewer consumers) will adopt OpenAI's o3 at wide scale, because of concerns about price and robustness relative to that price.
Deadline passed Evidence as of
Limitation: This record does not show how widely o3 was adopted in 2025. No evidence on its adoption is part of this record.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Sora will continue to have trouble with physics.”
Our reading: Filed under 'Medium confidence': at the end of 2025, OpenAI's Sora will still have trouble with physics.
Deadline passed Evidence as of
Limitation: The only evidence shown is OpenAI's own description, not an independent test.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“2025 could well be the year in which valuations for major AI companies start to fall.”
Our reading: Valuations of major AI companies could start to fall in 2025.
No horizon
Qualifier: filed under medium confidence
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Companies will continue to experiment with AI, but adoption to production-grade systems scaled out in the real-world will continue to be tentative.”
Our reading: With medium confidence, by the end of 2025 companies will continue to experiment with AI, but adoption of production-grade systems scaled out in the real world will continue to be tentative.
No horizon
Qualifier: filed under medium confidence
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Instead we may see models like o1 that are quite good at many tasks for which high-quality synthetic data can be created, but in other domains only incrementally better than GPT-4.”
Our reading: In 2025 we may see o1-like models good at synthetic-data-friendly tasks but only incrementally better than GPT-4 elsewhere.
No horizon
Qualifier: tasks where high-quality synthetic data can be created
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“My 2023 prediction that OpenAI might someday be seen as the WeWork of AI has not been confirmed, but they are far from out of the woods.”
Our reading: His 2023 prediction that OpenAI might someday be seen as the WeWork of AI has not been confirmed, but OpenAI is far from out of the woods.
No horizon Evidence as of
Limitation: This finding does not establish that OpenAI was financially secure or that the concern was unreasonable.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Sora will continue to have trouble with physics.”
Our reading: With medium confidence, by the end of 2025 Sora will continue to have trouble with physics.
No horizon
Qualifier: filed under medium confidence
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Few companies (and even fewer consumers) will adopt o3 at wide scale because of concerns about price and robustness relative to that price.”
Our reading: With medium confidence, by the end of 2025 few companies, and even fewer consumers, will adopt o3 at wide scale because of concerns about price and robustness relative to that price.
No horizon
Qualifier: because of concerns about price and robustness relative to that price
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Long term security, especially at high valuations, is this far from assured.”
Our reading: OpenAI's long-term security, especially at high valuations, is far from assured.
No horizon
Qualifier: especially at high valuations
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Technical moat will continue to be elusive. Instead, there will be more convergence on broadly similar models, across both US and China; some systems in Europe will catch up to roughly the same place.”
Our reading: With medium confidence, by the end of 2025 a technical moat will continue to be elusive, with more convergence on broadly similar models across the US and China, and some European systems catching up to roughly the same place.
No horizon
Qualifier: filed under medium confidence
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Neurosymbolic AI will become much more prominent.”
Our reading: With medium confidence, by the end of 2025 neurosymbolic AI will become much more prominent.
No horizon
Qualifier: filed under medium confidence
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“o3 has been previewed but not released likely will likely be quite expensive.)”
Our reading: OpenAI's o3 will likely be quite expensive when released.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“We may well see a large-scale cyberattack in which Generative AI plays an important causal role, perhaps in one of the four ways discussed in a short essay of mine that will appear shortly in Politico.”
Our reading: In 2025 there may be a large-scale cyberattack in which generative AI plays an important causal role.
No horizon
Qualifier: filed under low confidence
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Commercial artists and voiceover actors have perhaps been the hardest hit so far.”
Our reading: Commercial artists and voiceover actors have perhaps been hardest hit by AI so far.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I wouldn’t be shocked if none were reliably solved by the end of the year.”
Our reading: Possibly none of the Marcus-Brundage tasks will be reliably solved by end of 2025.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“(Google’s Veo 2 seems to be better but I have not been able to experiment with it, and suspect that changes of state and the persistence of objects will still cause problems;”
Our reading: Marcus suspects, without having tested it, that changes of state and the persistence of objects will still cause problems for Google's Veo 2, though it seems better than Sora.
No horizon
Qualifier: has not been able to experiment with it
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“There could continue to be no “GPT-5 level” model (meaning a huge, across the board quantum leap forward as judged by community consensus) throughout 2025.”
Our reading: There could be no GPT-5 level model, meaning a huge across-the-board leap by community consensus, throughout 2025.
No horizon
Qualifier: filed under low confidence
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“AI will presumably solve all of these someday, of course, but current techniques have thus far not been adequate.”
Our reading: AI will presumably eventually solve compositionality, factuality and reasoning, but current techniques have not been adequate.
No horizon
Qualifier: someday
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Both of us agree that the achievement of 8 out of 10 of these tasks would be strong evidence for the G (General) in Artificial General Intelligence having been achieved.”
Our reading: Marcus and Brundage agree that achieving 8 of the 10 listed tasks would be strong evidence that the 'General' in Artificial General Intelligence had been achieved.
No horizon
Qualifier: jointly agreed with Miles Brundage
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Because of the pretraining, and lack of comparability, what was actually shown was disappointing.”
Our reading: Because of the pretraining and the lack of comparability with humans, what o3 actually showed on ARC was disappointing.
No horizon
Qualifier: because of the pretraining and lack of comparability
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The media should be asking hard questions, not fanning hype.”
Our reading: The media should ask hard questions about AI instead of fanning hype.
No horizon
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Until there is considerable external scientific scrutiny (so far there has been none), we won’t really know exactly what the o3 advance is or how important it is.”
Our reading: Until o3 receives considerable external scientific scrutiny (so far it has had none), we won't really know exactly what the o3 advance is or how important it is.
No horizon
Only if: until there is considerable external scientific scrutiny
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“As the scientist Adan Becerra, PhD put it (and Chollet publicly agreed) the best thing would have been to present data for the “base model” without the pretraining.”
Our reading: OpenAI should have presented ARC results for the o3 base model without the pretraining.
No horizon
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The video should have been much clearer about what was actually tested and what was actually trained.”
Our reading: OpenAI's o3 announcement video should have been much clearer about what was tested and what was trained.
No horizon
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“As NYU prof Brenden Lake pointed out, the test should never have been called ARC-AGI.”
Our reading: The ARC test should never have been named ARC-AGI.
No horizon
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“People in the media probably shouldn’t even joke about o3 being AGI.”
Our reading: People in the media probably shouldn't even joke about o3 being AGI.
No horizon Evidence as of
Limitation: Only this normative sentence is assessed. Nearby factual claims in the post are not covered.
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Spending a quarter trillion dollars in order to have something 1000 times more expensive than average humans is maybe not the way to go.”
Our reading: Spending a quarter trillion dollars to get systems 1000x more expensive than humans is maybe not the right approach.
No horizon
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Until lots of people get to try o3 on many different kinds of tasks, we should not assume that it is reliable.”
Our reading: People should not assume o3 is reliable until many people have tested it on many kinds of tasks.
No horizon
Qualifier: until many people test it on many kinds of tasks
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I wish that others would give lines in the sand in advance; instead, they are likely to craft definitions post hoc, or not give them at all.”
Our reading: Those claiming AGI are likely to craft definitions post hoc or give none at all.
No horizon
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“And I am saying, as every roboticist on the planet knows, that we shouldn’t take demos seriously, until purported products are actually released and subjected to outside scrutiny.”
Our reading: AI demos should not be taken seriously until the products are released and subjected to outside scrutiny.
No horizon
Qualifier: until products are released and externally scrutinized
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Furthermore, the impact of all this on the environment could be immense.”
Our reading: The environmental impact of such high-compute AI systems could be immense.
No horizon
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“As in O3 may work for problems where massive data augmentation is possible, but not in others where it’s not.”
Our reading: o3 may work on problems allowing massive data augmentation but not on problems that do not.
No horizon
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I couldn’t agree more. Yesterday’s demo, impressive as it was, doesn’t change that.”
Our reading: The o3 demo does not change the view that AGI requires true logical reasoning rather than statistical tricks.
No horizon
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Nothing in the o3 presentation yesterday suggested to me that this decades-old core problem has been remotely solved.”
Our reading: Nothing in the o3 presentation suggested to Marcus that the decades-old core problem of neural networks generalizing better within distribution than outside it has been remotely solved.
No horizon Evidence as of
Limitation: This outcome makes no judgment about whether the presentation established a general solution to distribution shift.
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: Can’t be checkedSaid
Ed Zitron Conditional forecast
“You talking about the truth behind generative AI, or the harms of Facebook, or the gratuitous destruction of Google Search will change things, because these people are unprepared for a public that knows both what they’ve done and their sickening, loathsome, selfish and greedy intentions.”
Our reading: Public discussion of the truth behind generative AI and other tech harms will change things because tech leaders are unprepared for an informed public.
No horizon
Only if: if the public talks about the truth behind generative AI and other harms
Source: Never Forgive Them
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“This has irreparably damaged the internet in ways I believe few fully understand.”
Our reading: The flood of AI-generated content has irreparably damaged the internet.
No horizon
Source: Never Forgive Them
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“These companies unleashed generative AI on the world — or, in the case of Microsoft, facilitated its ascendency — without any consideration of what that would mean for the Internet as an ecosystem.”
Our reading: Google and Microsoft released or facilitated generative AI without considering its effect on the internet ecosystem.
No horizon
Source: Never Forgive Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Be the person who tells people that Sam Altman burns $5 billion a year on unsustainable software that destroys the environment and is built upon the large-scale larceny of creative works because he's desperate for power.”
Our reading: Readers should tell others that Sam Altman burns $5 billion a year on unsustainable, environmentally destructive software built on stolen creative works.
No horizon
Source: Never Forgive Them
Assessment: Can’t be checkedSaid
Ed Zitron Rhetoric
“Sam Altman is a con artist, a liar, and a sleazy carnival barker who would burn our planet to the ground, steal from millions of people and burn billions of dollars in pursuit of power, and I believe the same can be said of people like Dario Amodei of Anthropic and Mustafa Suleyman of Microsoft.”
Our reading: Sam Altman is a con artist, a liar, and a sleazy carnival barker who would burn the planet to the ground, steal from millions of people, and burn billions of dollars in pursuit of power.
No horizon
Source: Never Forgive Them
Assessment: Can’t be checkedSaid
Ed Zitron Rhetoric
“Sam Altman is a con artist, a liar, and a sleazy carnival barker who would burn our planet to the ground, steal from millions of people and burn billions of dollars in pursuit of power, and I believe the same can be said of people like Dario Amodei of Anthropic and Mustafa Suleyman of Microsoft.”
Our reading: Zitron believes the same can be said of people like Dario Amodei of Anthropic and Mustafa Suleyman of Microsoft: that they are con artists, liars, and sleazy carnival barkers who would burn the planet to the ground, steal from millions of people, and burn billions of dollars in pursuit of power.
No horizon
Qualifier: I believe the same can be said
Source: Never Forgive Them
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The problems by now seem clear. Suchir was right to consider them.”
Our reading: The copyright problems by now seem clear, and Suchir Balaji was right to consider them.
No horizon
Qualifier: seem
Source: Generative AI’s Continuing Copyright Problems, an Essay in Memory of Suchir Balaji, 1998 - 2024
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Society needs to do more to protect people like Suchir Balaji.”
Our reading: Society should do more to protect AI whistleblowers like Suchir Balaji.
No horizon
Source: Generative AI’s Continuing Copyright Problems, an Essay in Memory of Suchir Balaji, 1998 - 2024
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“It is unfortunate that CA SB-1047’s whistleblower protections were vetoed.”
Our reading: It is unfortunate that the whistleblower protections in California SB-1047 were vetoed.
No horizon
Source: Generative AI’s Continuing Copyright Problems, an Essay in Memory of Suchir Balaji, 1998 - 2024
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“I wouldn’t take seriously any of its anthropomorphic language about LLMs “knowing” or having scheming or having intentions to deceive, etc.”
Our reading: Anthropomorphic language about LLMs knowing, scheming, or intending to deceive should not be taken seriously.
No horizon
Qualifier: in the Apollo Research scheming paper
Source: Humanity’s “Oh shit!” AI moment?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“They more power we give them (but hooking up them to agents, people, etc), the more risk we are incurring.”
Our reading: Giving LLMs more power, such as hooking them up to agents, increases risk.
No horizon Evidence as of
Limitation: The only evidence item is one company's report on misuse of its own models. It records incidents and measures nothing about how harm scales with the power granted.
Source: Humanity’s “Oh shit!” AI moment?
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Even if machines don’t ever rise up a la the Terminator, a bad actor could still leverage LLMs to cause massive mayhem. Possibly soon.”
Our reading: A bad actor could leverage LLMs to cause massive mayhem, possibly soon.
No horizon
Qualifier: by a bad actor
Source: Humanity’s “Oh shit!” AI moment?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The important thing is that paper makes it very clear that nobody should ever take LLMs at their word.”
Our reading: Nobody should ever take LLMs at their word.
No horizon
Source: Humanity’s “Oh shit!” AI moment?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“The longer we delay in regulation, the more risk we are incurring.”
Our reading: Delaying AI regulation increases the risk society incurs.
No horizon Evidence as of
Limitation: The record has no outcome evidence at all, either for or against the claim.
Source: Humanity’s “Oh shit!” AI moment?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“There’s a lot of anthropomorphism here in this paper that I find hard to take, but it is in other ways a smart and carefully considered paper — and ought to be a wakeup call.”
Our reading: Apollo Research's paper on in-context scheming contains a lot of anthropomorphism he finds hard to take, but is in other ways a smart and carefully considered paper and ought to be a wakeup call.
No horizon
Qualifier: in other ways
Source: Humanity’s “Oh shit!” AI moment?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“That may leave all of us extremely vulnerable to this mess.”
Our reading: That Congress achieved little and that anti-regulation AI people like Andreessen are having a good run may leave all of us extremely vulnerable to this mess.
No horizon
Qualifier: may
Source: Humanity’s “Oh shit!” AI moment?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“I don’t doubt this, when stated so vaguely, but you can’t improve the engineering if you can’t identify the problems and weak points, such that you can address them.”
Our reading: You can't improve the engineering if you can't identify its problems and weak points so that you can address them.
No horizon Evidence as of
Limitation: This does not establish that the maxim is true or false.
Source: Casey (Newton) Strikes Out
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“But ignoring all the evidence that suggests that we won’t be there in the very near term does a disservice to his pretense towards objectivity.”
Our reading: Evidence suggests superintelligence will not be reached in the very near term.
No horizon
Qualifier: per available evidence
Source: Casey (Newton) Strikes Out
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Our best defense against the possible dangers of superintelligence must begin with a realistic appraisal of where we are and how much time we have.”
Our reading: Defending against superintelligence dangers must start with a realistic appraisal of current progress and timelines.
No horizon
Source: Casey (Newton) Strikes Out
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“A massive investment of hundreds of billions is unlikely to pay off.”
Our reading: The massive investment of hundreds of billions of dollars in AI scaling is unlikely to pay off.
No horizon
Source: Casey (Newton) Strikes Out
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The John Locke hypothesis that you can learn physics purely from sense data is failing, over, and over, and over again; this time perhaps (if I had to guess) to the tune of a billion dollars.”
Our reading: The author guesses that Sora's failed attempt to learn physics from data may have cost around a billion dollars.
No horizon
Qualifier: author's guess
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“One plausible explanation is that OpenAI has put a lot more work, with data augmentation and other such techniques, into math word problems than into image generation.”
Our reading: One plausible explanation for the difference in improvement is that OpenAI has put a lot more work, using data augmentation and other such techniques, into math word problems than into image generation.
No horizon
Qualifier: offered as one plausible explanation
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Relating language and the visual world robustly may require altogether new approaches to learning language.”
Our reading: Robustly relating language and the visual world may require altogether new approaches to learning language.
No horizon
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Until then the foundations of AI remain unsteady.”
Our reading: Until new approaches robustly relate language and vision, the foundations of AI remain unsteady.
No horizon
Qualifier: until new approaches to learning language exist
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“AI needs substantive regulation.”
Our reading: AI needs substantive regulation.
No horizon
Source: Hard-forked! Casey Newton’s distorted portrait of Gary Marcus and AI skepticism
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Many downsides are already heere, and potential misuse in the military seems imminent.”
Our reading: Military misuse of LLMs seems imminent, adding to downsides such as covert racism, deepfakes and discrimination that already exist.
No horizon
Source: Hard-forked! Casey Newton’s distorted portrait of Gary Marcus and AI skepticism