Claims that can’t be checked by Gary Marcus, page 19 of 24
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Claims that can’t be checked by Gary Marcus (2353)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 19 of 24: claims 1801 to 1900 of 2353, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Causal claim
“The tail thus appears to have wagged the dog—potentially imperiling the original mission, if there was any substance at all to the Board’s concerns.”
Our reading: The for-profit side of OpenAI appears to have overridden the nonprofit board, potentially putting the original mission at risk if there was any substance at all to the Board's concerns.
No horizon
Qualifier: appears to have
Assessment: Can’t be checkedSaid
Forecast
“If Altman returns and the board resigns as now seems likely”
Our reading: It seems likely that Sam Altman will return to OpenAI and the board will resign.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“How things play out might or might have major impact on the prospects of generative AI.”
Our reading: How the OpenAI leadership turmoil plays out might or might not have a major impact on the prospects of generative AI.
No horizon
Qualifier: "might or might have" read as an apparent typo for "might or might not", hedging both ways
Source: The biggest immediate lesson from the OpenAI meltdown
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The fact that the governance of one of the most visible AI companies in the world can change literally overnight should be a reminder that we can’t make our judgements about a company’s trustworthiness based simply on a vibe about their CEO.”
Our reading: The fact that the governance of one of the most visible AI companies in the world can change literally overnight should remind us that we can't judge a company's trustworthiness based simply on a vibe about its CEO.
No horizon
Qualifier: the governance of one of the most visible AI companies in the world can change literally overnight
Source: The biggest immediate lesson from the OpenAI meltdown
Assessment: Can’t be checkedSaid
Causal claim
“(In my view, without some sort of documentation, they wouldn’t have pulled the trigger).”
Our reading: In Marcus's view, the OpenAI board would not have fired Altman without some sort of documentation.
No horizon
Only if: without some sort of documentation
Assessment: Can’t be checkedSaid
Forecast
“The second is that nobody does something that dramatic without some kind of serious motivation; already it looks like the planned $86B valuation might not go through, per The Information.”
Our reading: According to The Information's reporting, it already looks like OpenAI's planned $86B valuation might not go through.
No horizon
Qualifier: per The Information
Assessment: Can’t be checkedSaid
Conditional forecast
“A lot of shareholders could be pissed; unless there was a smoking gun, the litigation could be endless.”
Our reading: A lot of shareholders could be angry, and unless there was a smoking gun, the litigation could be endless.
No horizon
Only if: unless there was a smoking gun
Assessment: Can’t be checkedSaid
Causal claim
“At some point the board came to believe this new company would undermine the nonprofit’s mission, and also discovered, perhaps e.g., from an email that someone got hold of, that he was not candid with them about important details—in ways that shattered trust.”
Our reading: Marcus guesses that the OpenAI board fired Altman because it believed his planned new venture would undermine the nonprofit's mission and found he had not been candid about important details, shattering trust.
No horizon
Qualifier: offered as a guess / theory of the case
Assessment: Can’t be checkedSaid
Observation
“The hubris of Spring of 2022 has to some small but detectable extent diminishing.”
Our reading: As of November 2023, the hubris of spring 2022 has diminished to some small but detectable extent.
No horizon
Qualifier: to some small but detectable extent
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The sooner we stop climbing the hill we are on, and start looking for new paradigms, the better.”
Our reading: AI research should stop pursuing the current LLM/scaling approach as soon as possible and look for new paradigms.
No horizon
Assessment: Can’t be checkedSaid
Observation
“GPT-4 (introduced in the intervening period) is surely better, but people have little by little started to recognize that scoring better on benchmarks is NOT the same as making foundational progress.”
Our reading: GPT-4 is surely better than the models of spring 2022, but people have little by little started to recognize that scoring better on benchmarks is not the same as making foundational progress.
No horizon
Qualifier: surely
Assessment: Can’t be checkedSaid
Causal claim
“Breaking news: the draft EU AI Act—the closest thing we have to AI regulation with real teeth—is in serious jeopardy, potentially a direct victim of regulatory capture, governments doing the bidding of those they are supposed govern.”
Our reading: The jeopardy facing the draft EU AI Act may be a direct result of regulatory capture, with governments doing the bidding of tech companies.
No horizon
Qualifier: potentially
Source: Crisis point in AI?
Assessment: Can’t be checkedSaid
Forecast
“Possible future: Literally every bad thing that comes out of generative AI-and there will may well be a lot—people will wonder: “maybe this wouldn’t have happened had Macron and Scholz not blocked the planet’s best effort at keeping the tech companies in check.””
Our reading: In a possible future, for literally every bad thing that comes out of generative AI (and there may well be a lot), people will wonder whether it might not have happened had Macron and Scholz not blocked the EU AI Act, which he calls the planet's best effort at keeping the tech companies in check.
No horizon
Qualifier: framed as a possible future
Source: Crisis point in AI?
Assessment: Can’t be checkedSaid
Conditional forecast
“If true self-driving is this hard, despite the amount of data that has been collected and amount of money invested ($100B+), any dreams of AGI happening soon are likely wildly unrealistic.”
Our reading: If true self-driving is this hard despite the data collected and $100B+ invested, any dreams of AGI happening soon are likely wildly unrealistic.
No horizon
Only if: if true self-driving is this hard, despite the data collected and $100B+ invested
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Forecast
“The whole self-driving cars industry is this likely to be viewed in a few years as an epic fail, crushed by outliers and AI that could not reason adequately.”
Our reading: Within a few years the self-driving car industry will likely be seen as an epic failure, defeated by outliers and AI's inadequate reasoning.
No horizon
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Forecast
“Driver-assist (a la Tesla) with humans in the loop may well stay, occasionally causing fatal accidents, given its lack of reliability”
Our reading: Driver-assist systems like Tesla's may persist while occasionally causing fatal accidents due to unreliability.
No horizon
Qualifier: humans in the loop
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Forecast
“We may see similar disappointments in safety-critical applications of LLMs (eg medicine), years of promise, but humans still for a very long time required in the loop, true autonomy difficult to achieve.”
Our reading: LLMs in safety-critical domains like medicine may disappoint, requiring humans in the loop for a very long time.
No horizon
Qualifier: safety-critical applications such as medicine
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Observation
“This is not true autonomy, but semi-autonomy. It might (I doubt it) save money, but it has literally nothing to do with AGI.”
Our reading: Remotely-assisted driving is semi-autonomy, not true autonomy; it might save money, though the author doubts it; and it has literally nothing to do with AGI.
No horizon
Qualifier: might save money (I doubt it)
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Forecast
“Quite possible nobody will turn a profit in remote driving anytime soon; quite possibly everybody is spending quite heavily on remote assist centers, in hopes of achieving true autonomy.”
Our reading: It is quite possible no company will profit from remote-assisted driving anytime soon.
No horizon
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Rhetoric
“No fantasy about AGI thus far has survived contact with the real world.”
Our reading: No AGI fantasy so far has held up against the real world.
No horizon
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Observation
“Waymo is probably better, but how much better is anybody’s guess.”
Our reading: Waymo is probably better, but how much better is anybody's guess.
No horizon
Qualifier: probably
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Forecast
“Remote driving itself is unlikely to work well enough as an endgame on public roads, because data communications just aren’t stable enough.”
Our reading: Remote driving is unlikely to work well enough as a long-term solution on public roads because data communications are not stable enough.
No horizon
Qualifier: on public roads
Source: Rethinking “driverless cars”
Assessment: Can’t be checkedSaid
Conditional forecast
“But it’s starting to feel like if the cars were actually left to their own devices, all hell would break loose.”
Our reading: It increasingly appears that Cruise cars would perform disastrously if left to operate without remote human help.
No horizon
Only if: if left without remote operators
Assessment: Can’t be checkedSaid
Conditional forecast
“Shareholders will certainly sue, and if it’s bad as it looks, I doubt that GM will continue the project, which was recently suspended.”
Our reading: If the situation is as bad as it looks, GM probably will not continue the Cruise driverless project.
No horizon
Only if: if it's as bad as it looks
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The State of California should demand answers immediately, and share them with the public.”
Our reading: California should immediately demand that driverless car companies disclose their remote operator use and make the answers public.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We really ought be a bit more patient about comparing risks and benefits.”
Our reading: We ought to be a bit more patient about comparing the risks and benefits of open source AI.
No horizon
Qualifier: a bit more patient
Source: Where we are right now on open source and potential AI risk.
Assessment: Can’t be checkedSaid
Observation
“There is a nonzero chance that they have put us all at massive risk, with zero oversight.”
Our reading: There is a nonzero chance that the company's open source AI push has put everyone at massive risk without oversight.
No horizon
Qualifier: nonzero chance
Source: Where we are right now on open source and potential AI risk.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The fact that a single company can unilaterally make this decision for all of humanity is terrifying.”
Our reading: It is terrifying that a single company can unilaterally make the open source AI decision for all of humanity.
No horizon
Scope: a single company
Source: Where we are right now on open source and potential AI risk.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“What we really need is a peer-review process by independent scientists; it still seems like the companies are free to do largely as we please. That’s not enough.”
Our reading: What is really needed is a peer-review process by independent scientists, because it still seems the order leaves companies free to do largely as they please, and that is not enough.
No horizon
Qualifier: it still seems companies are largely free
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The US Executive Order is an important first step in that direction”
Our reading: The US Executive Order is an important first step toward concrete implementation of AI policy.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“There’s a lot to like in the Executive Order. It’s fantastic that the US government is taking the many risks of AI seriously.”
Our reading: There is a lot to like in the White House AI Executive Order, and it is fantastic that the US government is taking the many risks of AI seriously.
No horizon
Qualifier: first impressions based on the fact sheet, before seeing the final order
Assessment: Can’t be checkedSaid
Forecast
“The Executive Order, as good as it is, may fall short.”
Our reading: The AI Executive Order may fall short despite its merits.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Not all AI uses foundational models at all (e.g., until recently probably few driverless car systems relied in a significant way on foundational models, and some probably still don’t), and it’s easy to add some other gadgets in and say your model is not a foundational model. Framing things as only pertaining to foundational models is plainly too narrow.”
Our reading: Framing the order's requirements as applying only to foundation models is plainly too narrow, because not all AI uses foundation models and it is easy to add other components and say a model is not a foundation model.
No horizon
Qualifier: e.g., until recently probably few driverless car systems relied significantly on foundation models, and some probably still don't
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We don’t need another position paper; we need concrete implementation.”
Our reading: AI governance needs concrete implementation rather than more position papers.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“On the whole, there is lot of good stuff around guidance, but I hope that we ultimately see more that is binding, with teeth.”
Our reading: US AI policy should ultimately include more binding rules with enforcement teeth beyond guidance.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“With respect to the example above, they will probably try to argue that their specific products don’t meet the risk threshold.”
Our reading: AI companies will probably argue their products don't meet the order's serious-risk threshold.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“The sooner we recognize that there is life beyond current approaches to AI, the faster we will get there.”
Our reading: The sooner the field recognizes that there are options beyond current AI approaches, the faster it will reach reliable self-driving, search and agents.
No horizon
Source: Why driverless cars might—or might not—be at the end of the road
Assessment: Can’t be checkedSaid
Causal claim
“The same by the way probably holds for generative search and personalized agents.”
Our reading: Generative search and personalized agents probably also need much smarter, reasoning, non-hallucinating AI that comes from fundamental research.
No horizon
Source: Why driverless cars might—or might not—be at the end of the road
Assessment: Can’t be checkedSaid
Forecast
“The Cruise announcement is likely to have a major impact, too.”
Our reading: GM's suspension of Cruise is likely to have a major impact on the self-driving industry.
No horizon
Source: Why driverless cars might—or might not—be at the end of the road
Assessment: Can’t be checkedSaid
Conditional forecast
“If that’s true, what works in a certain road in San Francisco might not generalize all that well to other places; every new jurisdiction (and every individual road within a jurisdiction) might need a lot of expensive work.”
Our reading: If Waymo depends on heavy road-specific customization, its approach might not generalize well beyond places like San Francisco, and each new jurisdiction or road might need a lot of expensive work.
No horizon
Only if: if the rumor that Waymo relies on heavy road-specific customization is true
Source: Why driverless cars might—or might not—be at the end of the road
Assessment: Can’t be checkedSaid
Conditional forecast
“If Alphabet pulls the plug on Waymo, it may be the end of driverless cars for now–but not forever.”
Our reading: If Alphabet shuts down Waymo, driverless cars may be finished for now, but not permanently.
No horizon
Only if: if Alphabet pulls the plug on Waymo
Source: Why driverless cars might—or might not—be at the end of the road
Assessment: Can’t be checkedSaid
Conditional forecast
“I agree that if Alphabet pull the plug, there will likely be a self-driving winter across the industry (though for now players like Aurora and Waabi may still have plenty of cash).”
Our reading: If Alphabet shuts down Waymo, the self-driving industry will likely enter a 'winter', though players like Aurora and Waabi may still have plenty of cash for now.
No horizon
Only if: if Alphabet pulls the plug on Waymo
Source: Why driverless cars might—or might not—be at the end of the road
Assessment: Can’t be checkedSaid
Observation
“And while every large tech company is making some effort around AI safety at this point, the collective sum of those efforts hasn’t yielded all that much.”
Our reading: The combined AI safety efforts of large tech companies have not yielded much.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Anthropic’s “Constitutional AI” idea seems to me to be clever and the best of the bunch, but still inadequate, riddled with the usual problems that LLMs face (reliability, excessive dependence on similarity to what is in the training set, hallucinations, unreliable reasoning, etc).”
Our reading: It seems to Marcus that Anthropic's Constitutional AI idea is clever and the best of the current AI safety ideas, but still inadequate, riddled with the usual LLM problems such as unreliability, excessive dependence on similarity to training data, hallucinations and unreliable reasoning.
No horizon
Qualifier: seems to me
Assessment: Can’t be checkedSaid
Observation
“What I noted in 2017 (at the time focusing on AI for medicine rather on AI safety per se) is that some problems in AI might be too complex for individual labs, and not of sufficient financial interest to the large AI companies like Google and Facebook (now Meta), ehich are after all, driven by profits rather than humanity. That’s still true.”
Our reading: Some AI problems may be too complex for individual labs and of insufficient financial interest to profit-driven large AI companies, which still holds true.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Don’t listen to them; enormous money is at stake.”
Our reading: People should not listen to those who deny there is any AI risk, such as Marc Andreessen, because enormous money is at stake.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We need to consider both current AI risk and AGI risk. Period.”
Our reading: We need to consider both current AI risk and AGI risk.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We need to stop worrying (just) about Skynet and robots taking over the world, and think a lot more about what criminals, including terrorists, might do with LLMs, and what, if anything, we might do to stop them.”
Our reading: AI risk discussion should focus much more on misuse of LLMs by criminals and terrorists, not just on AI takeover scenarios.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“The world can likely benefit enormously from AI, in ways that extend far beyond what we have already seen in recent months with productivity, but anyone with eyes can see that there are risks, too.”
Our reading: The world can likely benefit enormously from AI, in ways extending far beyond the productivity gains seen in recent months, though there are clearly risks too.
No horizon
Qualifier: can (potential benefit, not a prediction that it will happen)
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Count me among those who continue to favor a CERN for AI safety and Global Safety, so long as we balance the portfolio of risks we are trying to address.”
Our reading: A CERN for AI safety and global AI safety governance should be created, provided they address a balanced range of risks.
No horizon
Qualifier: balance the portfolio of risks addressed
Assessment: Can’t be checkedSaid
Forecast
“There are over 60 elections around the world in 2024; the chance that at least one will be swayed by deep fakes is high.”
Our reading: It is likely that at least one of the 60+ elections worldwide in 2024 will be swayed by deepfakes.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That seems like a sensible idea; indeed, it is a crude version of the kind of hybrid neuro-symbolic system that we have been advocating for many years.”
Our reading: Having GPT-4 use external tools such as the Code Interpreter and Wolfram Alpha plug-ins seems a sensible idea, and it is a crude version of the hybrid neuro-symbolic system the authors have advocated for many years.
No horizon
Qualifier: seems like a sensible idea
Source: Getting GPT to work with external tools is harder than you think
Assessment: Can’t be checkedSaid
Conditional forecast
“Some of these problems may fixable. With enough resources OpenAI or Wolfram Alpha could fix some of the problems in the interfaces.”
Our reading: With enough resources, OpenAI or Wolfram Alpha could fix some of the interface problems.
No horizon
Only if: with enough resources
Source: Getting GPT to work with external tools is harder than you think
Assessment: Can’t be checkedSaid
Conditional forecast
“All that said, much more fundamental problems remain and, until those problems are addressed, the ability of AI to apply mathematics to real world situation will remain limited.”
Our reading: Until fundamental problems are addressed, AI's ability to apply mathematics to real-world situations will remain limited.
No horizon Evidence as of
Only if: until fundamental problems are addressed
Source: Getting GPT to work with external tools is harder than you think
Assessment: Can’t be checkedSaid
Forecast
“We don’t claim that this problem is unsolvable, but addressing it may require major innovation.”
Our reading: Getting LLMs to reliably use external tools may require major innovation, though it is not necessarily unsolvable.
No horizon
Source: Getting GPT to work with external tools is harder than you think
Assessment: Can’t be checkedSaid
Causal claim
“A true understanding of elementary mathematics and its uses requires a deeper view.”
Our reading: True understanding of elementary math requires more than LLM token prediction or Wolfram-style symbol manipulation.
No horizon
Source: Getting GPT to work with external tools is harder than you think
Assessment: Can’t be checkedSaid
Forecast
“It may get fixed, with a band-aid of some sort or another, but we are confident a zillion other pictures like it will probably yield similar errors.”
Our reading: Even if this specific kitchen example gets patched, many other similar pictures will probably still produce similar errors in multimodal LLMs.
No horizon
Qualifier: even if this specific example is patched
Source: Hello, Multimodal Hallucinations
Assessment: Can’t be checkedSaid
Observation
“GPT-v (the visual version of GPT) can use words like refrigerator and counter, but that doesn’t mean it actually knows what these things mean..”
Our reading: GPT-4V can use words like refrigerator and counter, but that does not show it knows what they mean.
No horizon
Source: Hello, Multimodal Hallucinations
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The whole point of Rebooting AI was that the AI of 2018 didn’t actually understand how the world works, and that we couldn’t trust AI that didn’t. We stand fully by what we said.”
Our reading: The AI of 2018 did not actually understand how the world works, AI that does not understand how the world works cannot be trusted, and the authors fully stand by both points from Rebooting AI.
No horizon
Source: Hello, Multimodal Hallucinations
Assessment: Can’t be checkedSaid
Rhetoric
“The old systems were lost in a sea of correlations between words; the new system are lost in a sea of correlations between words and pixels.”
Our reading: Multimodal models, like text-only ones before them, depend on correlations (now between words and pixels) instead of understanding.
No horizon
Source: Hello, Multimodal Hallucinations
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“And 70% would be great for free throws; for multiplication it sucks.”
Our reading: Around 70% accuracy is unacceptable for multiplication, which requires exact answers.
No horizon
Source: “Math is hard” — if you are an LLM – and why that matters
Assessment: Can’t be checkedSaid
Forecast
“Some kind of hybrid may well work, but LLMs on their own remain stuck.”
Our reading: Some kind of hybrid system may well succeed at reliable math where pure LLMs fail.
No horizon
Source: “Math is hard” — if you are an LLM – and why that matters
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That, in a nutshell, is why we should never trust pure LLMs; even under carefully controlled circumstances with massive amounts of directly relevant data, they still never really get even the most basic linear functions.”
Our reading: Pure LLMs should never be trusted.
No horizon
Source: “Math is hard” — if you are an LLM – and why that matters
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It’s certainly plausible that some of the prerequisites of AGI may have been achieved, but considerable overreach to presume that all of “the most important ones” have already been invented.”
Our reading: It is certainly plausible that some prerequisites of AGI may have been achieved, but it is considerable overreach to presume that all of the most important ones have already been invented.
No horizon
Qualifier: certainly plausible that some ... may have been achieved
Assessment: Can’t be checkedSaid
Forecast
“Maybe what’s required already exists, but more likely, we are still at least a few paradigm shifts away .”
Our reading: AGI is more likely than not still at least a few paradigm shifts away.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The problem with the paper though isn’t just its hostility towards critics, or the way in which it lowers the bar, it’s that its key arguments don’t go through.”
Our reading: The problem with the Agüera y Arcas and Norvig paper is not just its hostility toward critics or the way it lowers the bar for AGI, but that its key arguments don't go through.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The fact that LLMs trained on the task of next-token prediction can also do a pretty good job at a variety of natural language tasks, and, very recently, a considerably less good job of tasks that combine text with images, as emphasized by Agüera y Arcas and Norvig is certainly interesting, but it doesn’t justify the claim that they constitute “general intelligence”.”
Our reading: LLMs trained on next-token prediction doing a pretty good job at a variety of natural language tasks, and very recently a considerably less good job at tasks combining text with images, is certainly interesting but does not justify the claim that they constitute general intelligence.
No horizon
Qualifier: very recently
Assessment: Can’t be checkedSaid
Forecast
“In AI, no single technique is likely to be a silver bullet, now or ever.”
Our reading: No single AI technique is likely ever to be a silver bullet.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But there is zero justification for claiming that the current technology has achieved general intelligence.”
Our reading: There is zero justification for claiming that the current technology has achieved general intelligence.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Current models trained on next-token prediction have remarkable abilities and remarkable weaknesses. The scope of those abilities and weaknesses of the current systems is not well understood.”
Our reading: Current next-token prediction models have remarkable abilities and weaknesses whose scope is not well understood.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“If GPT-5 did fail to please, or even if it were just deferred indefinitely (like so many promises that have been about driverless cars), it could have a rapid, deflationary effect on sky-high valuations.”
Our reading: If GPT-5 disappoints or is indefinitely deferred, AI valuations could deflate rapidly.
No horizon
Only if: if GPT-5 disappoints or is deferred indefinitely
Assessment: Can’t be checkedSaid
Observation
“They might have decided that GPT-5, if it was simply a bigger version of GPT-4, would not meet expectations, and that it wasn’t worth spending the hundreds of millions of dollars required, if the outcome would only turn out to be disappointing, or even embarrassing.”
Our reading: OpenAI might have decided, perhaps after early proof-of-concept tests by May 2023, that GPT-5, if simply a bigger GPT-4, would not meet expectations and was not worth the hundreds of millions of dollars required if the outcome would be disappointing or embarrassing.
No horizon
Qualifier: if GPT-5 was simply a bigger version of GPT-4
Assessment: Can’t be checkedSaid
Conditional forecast
“People might even start to realize that building truly reliable, trustworthy AI is actually really, really hard.”
Our reading: If GPT-5 failed to please or were deferred indefinitely, people might even start to realize that building truly reliable, trustworthy AI is really, really hard.
No horizon
Only if: if GPT-5 failed to please or were deferred indefinitely
Assessment: Can’t be checkedSaid
Observation
“Another theory is that OpenAI realized how expensive it would be either to train the model or to run it, and wasn’t sure they could make a profit given those costs. That still seems viable.”
Our reading: It is plausible that OpenAI was unsure it could profit from GPT-5 given training and inference costs.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“If Yann LeCun and I are correct, AI will need genuinely new paradigms to get to the next level in AI, and OpenAI may be no closer to that than anyone else.”
Our reading: If Marcus and LeCun are correct, AI will need genuinely new paradigms to get to the next level, and OpenAI may be no closer to that than anyone else.
No horizon
Only if: if Yann LeCun and Marcus are correct
Assessment: Can’t be checkedSaid
Forecast
“Few have risen as quickly, but there’s no guarantee that OpenAI can maintain their stratospheric trajectory.”
Our reading: OpenAI may not be able to maintain its rapid growth trajectory.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It's mean *working* for a positive future, not just declaring that one is inevitable and hoping for the best.”
Our reading: A positive technological future has to be actively worked for, not just declared inevitable while hoping for the best.
No horizon
Source: Two kinds of Techno-Optimism
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It means working to recognize the risks, and working to address them, so that we can reach the positive outcomes that are promised, not casting those who recognize those risks as the enemy.”
Our reading: Techno-optimists should work to recognize and address technology's risks to reach its promised benefits, rather than treating people who point out those risks as enemies.
No horizon
Source: Two kinds of Techno-Optimism
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But being long-term optimistic about technology doesn't mean you have to ignore the short-term (or long-term) risks of technology.”
Our reading: Being long-term optimistic about technology does not mean one has to ignore the short-term or long-term risks of technology.
No horizon
Scope: for someone who is long-term optimistic about technology
Source: Two kinds of Techno-Optimism
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Our focus should be on figuring on how to build AI that can represent and reason about *values*, rather than simply perpetuating past data”.”
Our reading: AI research should focus on building systems that can represent and reason about values instead of just perpetuating past data.
No horizon
Source: Race, statistics, and the persistent cognitive limitations of DALL-E
Assessment: Can’t be checkedSaid
Observation
“It’s that it can’t separate the world from the statistics of its dataset. DALL-E does not have a cognitive construct of a doctor or a patient or a human being or an occupation or medicine or race or egalitarianism or equal opportunity or any of that.”
Our reading: DALL-E 3 cannot distinguish the world from its training data statistics and has no cognitive concepts of doctors, patients, race, or similar.
No horizon
Source: Race, statistics, and the persistent cognitive limitations of DALL-E
Assessment: Can’t be checkedSaid
Causal claim
“But the level of actual cognitive comprehension of here is low. The modest number of properly labeled examples is presumably swamped by all the billions of watch advertisements.”
Our reading: Multimodal ChatGPT has low real comprehension, and the few correctly labeled clock examples are probably drowned out by billions of watch ads showing the canonical time.
No horizon
Source: Race, statistics, and the persistent cognitive limitations of DALL-E
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Blind fealty to the statistics of arbitrary data is not AGI. It’s a hack. And not one that we should want.”
Our reading: Blind fealty to the statistics of arbitrary data is not AGI; it is a hack, and not one we should want.
No horizon
Source: Race, statistics, and the persistent cognitive limitations of DALL-E
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“What one really needs, and no one yet knows how to build, is a system that can derive semantics of wholes from their parts as a function of their syntax.”
Our reading: AI needs systems that derive the meaning of wholes from their parts according to syntax, which no one yet knows how to build.
No horizon
Source: Race, statistics, and the persistent cognitive limitations of DALL-E
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“but we are in complete agreement that we are at a turning point with enormous uncertainty, and that we need to make the right choices now.”
Our reading: AI is at a turning point of enormous uncertainty, and society needs to make the right choices now.
No horizon
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Observation
“To be fair, large language models aren’t just looking things up, but the idea that a good prediction of next word necessarily implies understanding is fallacious.”
Our reading: Good next-word prediction by LLMs does not necessarily imply understanding.
No horizon Evidence as of
Limitation: This does not reject Marcus's argument or endorse Hinton's.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Observation
“Of course, I don’t think these machines are conscious, either.”
Our reading: Current AI machines are not conscious.
No horizon
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Observation
“But I don’t we are all that close to machines that are more intelligent than us”
Our reading: We are not very close to machines that are more intelligent than humans.
No horizon
Qualifier: partly definitional
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We are giving way too much authority to machines that we can’t control.”
Our reading: We are giving way too much authority to machines that we can't control.
No horizon
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Since they can’t reliably solve them, I don’t think we should attribute “understanding” to them, at least not in any remotely deep sense of the word understanding.”
Our reading: We should not attribute deep understanding to generative AI systems because they cannot reliably solve basic problems like family-relationship inference.
No horizon
Qualifier: because they cannot reliably solve basic inference problems
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“this is one of the reasons laws should be written to make machines disclose the fact that they are machines.”
Our reading: Laws should require machines to disclose that they are machines.
No horizon
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Observation
“I am much more worried about bad actors deliberately misusing AI, than machines deliberately wanting to take over. But Geoff’s right that we can’t fully rule it out either.”
Our reading: Marcus is much more worried about bad actors deliberately misusing AI than about machines deliberately wanting to take over, though the possibility of a takeover cannot be fully ruled out.
No horizon
Qualifier: cannot fully rule it out
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: Can’t be checkedSaid
Observation
“Finding that some stuff correlates with space or time doesn’t mean that stuff genuinely represents space or time.”
Our reading: The correlations with space and time in Gurnee and Tegmark's findings do not show that LLMs genuinely represent space or time.
No horizon
Source: Muddles about Models
Assessment: Can’t be checkedSaid
Observation
“Of course as Fei Fei Li correctly pointed out on X, it all depends on what you mean by a “model”, but here’s the crux, same as it ever was: correlations aren’t causal, semantic models.”
Our reading: Correlations found inside LLMs are not causal, semantic world models.
No horizon
Source: Muddles about Models
Assessment: Can’t be checkedSaid
Forecast
“And here is why all this matters: safety, which is what Tegmark really cares about, is not going to emerge anytime soon, and probably ever, from hit-or-miss systems that substitute correlation for a factual, structured, interrogable representations of the world.”
Our reading: AI safety will not emerge anytime soon, and probably never, from correlation-based systems that lack structured, interrogable representations of the world.
No horizon
Qualifier: systems that substitute correlation for structured representations
Source: Muddles about Models
Assessment: Can’t be checkedSaid
Observation
“I would conjecture that you can’t say to ChatGPT, let’s play a game of chess and then (say) 37 moves in pause, and get it to reliably answer how many white bishops are left on the board, nor to explain what happened to some pawn that had been taken a few moves earlier.”
Our reading: Marcus conjectures that ChatGPT cannot reliably say how many white bishops remain, or what happened to a captured pawn, partway through a chess game.
No horizon
Qualifier: conjecture
Source: Muddles about Models
Assessment: Can’t be checkedSaid
Conditional forecast
“To encapsulate the heart of what Ernie Davis and I said at length in our 2019 book Rebooting AI, AI is unlikely to ever be trustworthy if its makers continue to skip the step of having explicit cognitive models of the world.”
Our reading: AI is unlikely ever to be trustworthy if its makers keep skipping explicit cognitive models of the world.
No horizon
Only if: if makers continue to skip explicit cognitive models of the world
Source: Muddles about Models
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The worst part is not that every single sentence contains at least one lie, but that the whole thing sounds detailed and real. To someone who wasn’t following matters closely, it might well all sound plausible.”
Our reading: The worst part of Bing's false answer is not that every sentence contains at least one falsehood but that the whole thing sounds detailed and real, so that to someone who wasn't following matters closely it might well all sound plausible.
No horizon
Scope: to someone who wasn't following matters closely
Source: Seven lies in four sentences
Assessment: Can’t be checkedSaid
Rhetoric
“Ezra and I harped on their propensity to generate bullshit because bullshit was and is their fundamental product.”
Our reading: Bullshit was and still is the fundamental product of LLMs.
No horizon
Source: Seven lies in four sentences
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“we really ought to commit to building AI that can.”
Our reading: The field should commit to building AI that can distinguish facts from fiction.
No horizon
Source: Seven lies in four sentences