Claims that can’t be checked, page 27 of 37
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims that can’t be checked (3643)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 27 of 37: claims 2601 to 2700 of 3643, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“But if the White House treats tech CEOs like rock stars, those rock stars are going to call the shots.”
Our reading: If the White House treats tech CEOs like rock stars, those CEOs will determine AI policy.
No horizon
Only if: if the White House treats tech CEOs like rock stars
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Every one of us should be worried.”
Our reading: Every one of us should be worried about Washington remaining in Silicon Valley's thrall, elections under siege, widespread deepfake scams, and no AI law having passed.
No horizon
Scope: every one of us
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Schumer has had something like nine closed door meetings on AI, but (to my knowledge) yet to bring a single AI-related piece of legislation to the Senate floor. If he doesn’t, we are sitting ducks.”
Our reading: If Schumer does not bring AI-related legislation to the Senate floor, we will be sitting ducks, meaning highly vulnerable.
No horizon
Only if: if Schumer doesn't bring AI-related legislation to the Senate floor
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Washington is supposed to be a counterweight, to keep the captain of industries honest. So far, they haven’t done enough.”
Our reading: So far, Washington has not done enough as a counterweight to keep the captains of industry honest.
No horizon
Qualifier: so far
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The premise of the book is that Silicon Valley is increasingly controlling our lives, and indeed the entire world, and that Washington (seat of the US government) is too much in their thrall to stand up tall enough against it.”
Our reading: Silicon Valley is increasingly controlling our lives and the entire world, and Washington is too much in its thrall to stand up tall enough against it.
No horizon
Scope: our lives, and indeed the entire world
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“And that we need to act, now.”
Our reading: We need to act now to counter Silicon Valley's control.
No horizon
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Particularly since Silicon Valley is prone to hype, Washington needs independent scientists in the room.”
Our reading: Because Silicon Valley is prone to hype, Washington needs independent scientists involved in AI policy discussions.
No horizon
Qualifier: since Silicon Valley is prone to hype
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“One of the major themes of Taming Silicon Valley is the need for independent oversight; as the old proverb so rightly puts it, the fox can’t guard the henhouse.”
Our reading: AI needs independent oversight rather than self-regulation by the tech industry.
No horizon
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“It’s almost impossible to envision the independence we might need when nobody in Washington knows where they will be come January, and pots of gold await in the Bay Area.”
Our reading: It is almost impossible to envision the independence we might need in Washington when nobody there knows where they will be come January and lucrative jobs await in the Bay Area.
No horizon
Only if: when nobody in Washington knows where they will be come January, and pots of gold await in the Bay Area
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“We waited too long.”
Our reading: We waited too long to act against AI-generated misinformation and deepfakes in elections.
No horizon
Source: We can’t trust the fox to guard the henhouse, especially when it comes to AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Instead we need to wake up and realize that hallucinations are absolutely core to how LLMs work, and that we need new approaches based on different ideas.”
Our reading: The AI field needs new approaches based on ideas other than LLMs to address hallucinations.
No horizon
Qualifier: because hallucinations are core to LLMs
Source: Facing Facts
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“TL;DR: Don't console yourself with making something that superficially looks like human errors, if you aspire to AGI.”
Our reading: Those aiming for AGI should not take comfort in LLM errors superficially resembling human errors.
No horizon
Qualifier: if you aspire to AGI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Rather, the goal of AGI should be to build machines that can reliably reason and plan about a wide swathe of the world.”
Our reading: The goal of AGI should be machines that reliably reason and plan about a wide swathe of the world, not recreating humans.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“The field of AI will eventually do better, but probably with an AI that is structured differently, in which facts are first-class citizens, rather than something you hope you might get for free with enough data.”
Our reading: AI will eventually do better on hallucinations, probably via differently structured systems that treat facts as first-class rather than via more data.
No horizon
Qualifier: eventually
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“But even with perfect data, the stochastic reconstructive process would still produce some errors.”
Our reading: Even with perfect data, the stochastic reconstructive process LLMs use would still produce some errors.
No horizon
Qualifier: even with perfect data
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Gary Marcus is shaking his head, but cautiously optimistic that an increase of realism might lead to a renaissance of new approaches to AI.”
Our reading: An increase in realism about AI might lead to a renaissance of new approaches to AI.
No horizon
Only if: if realism about AI increases
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Meanwhile, and perhaps not coincidentally, the bubble deflation that I have been warning since August of has finally perhaps begun:”
Our reading: The AI bubble deflation may have begun as of April 2024.
No horizon
Qualifier: perhaps
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“So 2024 could be remembered as the high point in a kind of tulip-mania.”
Our reading: 2024 could be remembered as the peak of an AI tulip-mania.
No horizon
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Historians are going to be scratching their heads.”
Our reading: Historians will be puzzled by the AI industry's groupthink.
No horizon
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Strikingly, Meta is giving it away free, and the more deeply-trained version (still to come/still being trained) may well be in the general vicinity of GPT-4.”
Our reading: The more deeply-trained version of Meta's LLAMA-3, still being trained, may well be in the general vicinity of GPT-4.
No horizon
Scope: more deeply-trained version still to come/still being trained
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Dreamers who promise far more than they are likely to deliver are getting almost all the money, and (at least in AI) they are getting so much money for a path that it is unlikely to work that they are leaving too little oxygen for the exploration of new ideas that might work.”
Our reading: Dreamers who promise far more than they are likely to deliver are getting almost all the money, and, at least in AI, they are getting so much money for a path unlikely to work that too little oxygen is left for exploring new ideas that might work.
No horizon
Scope: at least in AI (applies to the crowding-out claim)
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Dreamers who promise far more than they are likely to deliver are getting almost all the money, and (at least in AI) they are getting so much money for a path that it is unlikely to work that they are leaving too little oxygen for the exploration of new ideas that might work.”
Our reading: The dominant, heavily funded current AI approach is unlikely to work.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“There is also a hostility in some parts of Silicon Valley right now to regulation. This a mistake.”
Our reading: The hostility to regulation in some parts of Silicon Valley right now is a mistake.
No horizon
Scope: some parts of Silicon Valley
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Right now I think the current net effect of venture capital is not great.”
Our reading: Marcus thinks that right now the net effect of venture capital is not great.
No horizon
Qualifier: I think
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“I don’t see how we get to a positive AI future if we sweep all the problems under the rug.”
Our reading: Marcus does not see how a positive AI future can be reached if all of AI's problems are swept under the rug.
No horizon
Only if: if we sweep all the problems under the rug
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“But we can only get there if we can face the limitations and risks honestly.”
Our reading: Technologies that improve the world can only be reached if we can face their limitations and risks honestly.
No horizon
Only if: unless we can face the limitations and risks honestly
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I doubt that anyone with a short leash can achieve the breakthroughs we will need, if we are to get to the optimistic future so many of us would like to see.”
Our reading: AI efforts funded on a short leash are unlikely to achieve the breakthroughs needed for an optimistic AI future.
No horizon
Qualifier: with a short funding leash
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I think Khosla’s timing (all this by 2049) might be off, perhaps by decades, but I don’t see why we can’t get to most of this eventually.”
Our reading: Khosla's predicted technological advances may not all arrive by 2049, perhaps being decades late, though most could be achieved eventually.
No horizon
Qualifier: perhaps off by decades
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I don’t think we are any closer to a machine with the wherewithal or the motivation to annihilate the human species than we were a year ago.”
Our reading: He does not think we are any closer to a machine with the wherewithal or the motivation to annihilate the human species than we were a year ago.
No horizon
Qualifier: I don't think
Source: TED, AI, and the test of time
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I love Khan Academy, but continue to have doubts about whether LLMs are really the right tool for the next step of the job.”
Our reading: He continues to have doubts about whether LLMs are really the right tool for the next step of the job of transforming education.
No horizon
Qualifier: continues to have doubts
Source: TED, AI, and the test of time
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“Generative AI's energy demands are unsustainable, as are its compute demands”
Our reading: Generative AI's energy and compute demands are unsustainable.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“The answer is obvious: because they believe it'll make them money.”
Our reading: The tech industry is pushing generative AI on people because it believes generative AI will make it money.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“And when you have an industry piloted by people who don't make products for people, you don't create anything useful, and you're doomed to make the same stupid, obvious mistakes.”
Our reading: When an industry is piloted by people who don't make products for people, it doesn't create anything useful and is doomed to make the same stupid, obvious mistakes.
No horizon
Only if: an industry piloted by people who don't make products for people
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Even the claims of what generative AI has done need to be taken with a grain — or a spoonful — of salt.”
Our reading: Claims about what generative AI has achieved should be treated with heavy skepticism.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“Nothing — and I repeat, nothing — good will come from this.”
Our reading: Nothing good will come from Google's leadership shuffle that moved senior AI engineer Cheenu Venkatachary to lead the ranking and search quality teams.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“The generative AI boom has exactly the same stench as the metaverse, and it's happening for exactly the same reason — the people making the products are not building things for human beings, but to show the markets that they'd continue to grow.”
Our reading: Like the metaverse, the generative AI boom is driven by the need to show markets continued growth rather than to build things for people.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Every single tech company making a LLM is stealing, justifying it by using the previous model of the internet where everything published online was there for the taking and conflating access to content with ownership in the process.”
Our reading: Every single tech company making an LLM is stealing, justifying it with the earlier model of the internet in which everything published online was there for the taking, and conflating access to content with ownership.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“LLMs are a globally-perpetuated act of theft taking place in broad daylight, to the point that OpenAI has told England's House of Lords that "it would be impossible to train today's leading AI models without using copyrighted materials."”
Our reading: LLMs are a globally perpetuated act of theft taking place in broad daylight.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“OpenAI is no different. After all, it’s effectively a subsidiary of Microsoft.”
Our reading: OpenAI is effectively a subsidiary of Microsoft and, like the others, is a thief rather than an innovator.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“While OpenAI, Google and Meta would like to claim that these are "publicly-available" works that they are "training on," the actual word for what they're doing is "stealing."”
Our reading: Although OpenAI, Google and Meta call it training on 'publicly-available' works, the right word for what they are doing is 'stealing'.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Yet I believe that they're all disconnected from any actual value creation.”
Our reading: He believes Google, Meta, Microsoft and OpenAI are all disconnected from any actual value creation.
No horizon
Qualifier: I believe
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Ed Zitron Rhetoric
“I’d say it has the engineering ability of a first-year CompSci undergraduate, but that would be unfair to first-year CompSci undergraduates.”
Our reading: Devin's engineering ability is below that of a first-year computer science undergraduate.
No horizon
Source: They're Looting The Internet
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Both damning and correct. People who believe in LLMs have played themselves.”
Our reading: Bjarnason's LLMentalist analysis is correct, and people who believe in LLMs have fooled themselves.
No horizon
Source: LLMs aren’t very bright. Why are so many people fooled?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Gary Marcus thinks the whole world would be different if so many people weren’t so easily taken by the parlor tricks of generative AI.”
Our reading: Gary Marcus thinks the whole world would be different if so many people were not so easily taken in by the parlor tricks of generative AI.
No horizon
Qualifier: Gary Marcus thinks
Source: LLMs aren’t very bright. Why are so many people fooled?
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The second summary, deeply parallel with the first, outlines the process by which an unthinking LLM (without meaning to do so) cons many of its users.”
Our reading: An unthinking LLM, without meaning to do so, cons many of its users through a process that closely parallels a psychic's con.
No horizon Evidence as of
Limitation: The essay page shows no publication date. Its date is recorded as 2023 based on Marcus's note ('from last year'), and its available_at is the post date.
Source: LLMs aren’t very bright. Why are so many people fooled?
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Reliable, trustworthy AI is surely achievable, but we may need to go back to the drawing board to get there.”
Our reading: Reliable, trustworthy AI is surely achievable, but we may need to go back to the drawing board to get there.
No horizon
Qualifier: achievability stated as 'surely'
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“In the most extreme case, OpenAI’s $86 billion valuation could look in hindsight like a WeWork moment for AI.”
Our reading: In the most extreme case, OpenAI's $86 billion valuation could in hindsight look like a WeWork moment for AI.
No horizon
Qualifier: in the most extreme case
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“What really got me to thinking about all this was a graph from OpenAI a couple days ago, touting their latest model, GPT 4 Turbo, which I have always suspected was a failed attempt at GPT-5.”
Our reading: Marcus suspects GPT-4 Turbo was a failed attempt at building GPT-5.
No horizon
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“My gut sense is that we haven’t reached the real ceiling on MMLU yet, and that this is a genuine sign of diminishing returns.”
Our reading: The MMLU plateau is likely a genuine sign of diminishing returns rather than a ceiling effect.
No horizon
Qualifier: MMLU real ceiling not yet reached
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“The wall that I once warned about, in 2022, may finally be approaching.”
Our reading: LLM progress may be approaching the wall Marcus warned about in 2022.
No horizon
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“If we really have changed regimes, from rapid progress to diminishing returns, and hallucinations and stupid errors do linger, LLMs may never be ready for prime time.”
Our reading: If we really have changed regimes from rapid progress to diminishing returns, and hallucinations and stupid errors do linger, LLMs may never be ready for prime time.
No horizon
Only if: if we really have changed regimes from rapid progress to diminishing returns
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“But that place may be smaller; it is entirely possible that LLMs on their own will never live up to last year’s wild expectations.”
Our reading: LLMs' role may be smaller than hoped, and LLMs alone may never live up to 2023's wild expectations.
No horizon
Qualifier: LLMs on their own
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I could be persuaded that on some measures there was a doubling of capabilities for some set of months in 2020-2023, but I don’t see that case at all for the last 13 months.”
Our reading: Marcus could be persuaded that LLM capabilities doubled on some measures for some months in 2020-2023, but he does not see that case at all for the 13 months since GPT-4's release.
No horizon Evidence as of
Limitation: The packet contains no evidence items establishing a defined capability comparison.
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Whatever doubling there might have been, has perhaps come to an end.”
Our reading: Any doubling of LLM capabilities has perhaps come to an end.
No horizon
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“If two graphs I plotted are remotely correct, Mollick’s claim that “the rate of improvement in Large Language models show capabilities doubling every 5 to 14 months” is no longer holding true.”
Our reading: If his graphs are roughly correct, LLM capabilities are no longer doubling every 5 to 14 months.
No horizon Evidence as of
Limitation: This does not endorse Mollick's wording.
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Instead, as I warned in August, we may well be in for a correction.”
Our reading: The AI market may well be in for a correction.
No horizon
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“And of course advancing on benchmarks isn’t itself enough anyway; few benchmarks capture the complexity of the real world. Even if LLMs could max out on all existing benchmark, we might still have a long way to go.”
Our reading: Advancing on benchmarks is not enough by itself because few benchmarks capture the complexity of the real world, and even if LLMs could max out all existing benchmarks, we might still have a long way to go.
No horizon
Qualifier: even if LLMs could max out on all existing benchmarks
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Learn and master the basics of almost any new video game within a few minutes or hours, and solve original puzzles in the alternate world of that video game.”
Our reading: Marcus seriously doubts that, by the end of 2025, AI will be able to learn and master the basics of almost any new video game within a few minutes or hours and solve original puzzles in the alternate world of that video game.
Deadline passed Evidence as of
Limitation: This record does not show what AI systems could do with new video games in 2025. No evidence on that question is part of this record.
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Write cogent, persuasive legal briefs without hallucinating any cases”
Our reading: AI is unlikely by end of 2025 to write cogent, persuasive legal briefs without hallucinating any cases.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Watch previously unseen mainstream movie and be able to follow plot twists and know when to laugh, and be able to summarize it without giving away any spoilers or making up anything that didn’t actually happen”
Our reading: AI is unlikely by end of 2025 to watch an unseen mainstream movie, follow plot twists, know when to laugh, and summarize it without spoilers or fabrications.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“None seem to me to be within immediate reach.”
Our reading: None of the listed tasks seem within immediate reach of AI.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Right now there are likely hundreds, maybe thousands or even millions of such things.”
Our reading: As of April 2024 there are likely hundreds or more pieces of intellectual work that the best humans can do and AI cannot.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Walk into random kitchens and prepare a cup of coffee or make breakfast”
Our reading: AI is unlikely by end of 2025 to walk into random kitchens and prepare coffee or breakfast.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Babysit children in an unfamiliar homes and keep them safe”
Our reading: AI is unlikely by end of 2025 to babysit children in unfamiliar homes and keep them safe.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Drive off-road vehicles, without maps, across streams, around obstacles such as fallen trees, and so on.”
Our reading: AI is unlikely by end of 2025 to drive off-road vehicles without maps across streams and around obstacles.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Clean unfamiliar houses and apartments”
Our reading: AI is unlikely by end of 2025 to clean unfamiliar houses and apartments.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Learn to ride a mountain bike off-road through forest trails”
Our reading: AI is unlikely by end of 2025 to learn to ride a mountain bike off-road through forest trails.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Tend to the physical and psychological needs of an elderly or infirm person”
Our reading: AI is unlikely by end of 2025 to tend to the physical and psychological needs of an elderly or infirm person.
No horizon
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Reliably drive a car in a novel location that they haven’t previously encountered, even in the face of unusual circumstances like hand-lettered signs, without the assistance of other humans.”
Our reading: AI is unlikely by end of 2025 to reliably drive a car in novel locations with unusual circumstances without human assistance.
No horizon
Qualifier: without assistance of other humans
Source: Superhuman AGI is not nigh
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“In another bit of possible foreshadowing: GenAI is starting to get a tiny bit of Scarlet Letter feel.”
Our reading: Generative AI is starting to get a tiny bit of a 'Scarlet Letter' feel, which he presents as possible foreshadowing.
No horizon
Qualifier: a tiny bit
Source: 24 hours in AI news
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Another sign that the bubble may not last forever.”
Our reading: The generative AI investment bubble may not last forever, as signalled by some late-stage VC firms sitting out.
No horizon
Qualifier: based on some late-stage VCs sitting out the GenAI rush
Source: 24 hours in AI news
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Not one of these problems has been adequately addressed.”
Our reading: None of the problems listed (hallucinations, abstraction and reasoning, brittleness, data guzzling, etc.) has been adequately addressed.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“We must invest heavily in alternative approaches and stop funding a losing hand to the infinite degree, crowding everything out.”
Our reading: The field must invest heavily in alternatives to generative AI and stop pouring unlimited funding into it, which crowds out everything else.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“stop funding a losing hand to the infinite degree, crowding everything out.”
Our reading: We must stop funding a losing hand to the infinite degree, which crowds everything else out.
No horizon
Qualifier: to the infinite degree
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“a pile of unreliable systems that are unlikely to live up to the hype.”
Our reading: Current generative AI systems are unreliable and are unlikely to live up to the hype.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“All this happened upon the realization that their systems simply cannot succeed without even more data than the internet-scale data they have already been trained on.”
Our reading: The generative AI systems of major AI companies cannot succeed without even more data than the internet-scale data they were already trained on.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Mixing bogus chatbot stories in with the rest is a recipe for disaster, both for humans and for LLMs. It’s a downward spiral waiting to happen.”
Our reading: Mixing bogus chatbot stories in with the rest is a recipe for disaster for both humans and LLMs, a downward spiral waiting to happen.
No horizon
Scope: both for humans and for LLMs
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Gary Marcus doesn’t know how to it say it more clearly: fast, cheap, automated misinformation left unchecked threatens to undermine trust — and democracy itself.”
Our reading: Unchecked fast, cheap, automated misinformation threatens to undermine trust and democracy.
No horizon
Qualifier: left unchecked
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Bluntly, we risk falling directly into Putin’s hands:”
Our reading: We risk falling directly into Putin's hands.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Senator Schumer, please take note. We need to move on this, fast.”
Our reading: Congress needs to act quickly on AI misinformation, including labeling requirements.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“One of the (many) suggestions, almost the least we could ask for, is that we should require all AI-generated content to be labeled.”
Our reading: US law should require all AI-generated content to be labeled.
No horizon
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“The article is vague, likely because building Stargate is at this point somewhat theoretical.”
Our reading: The Information's article on Stargate is vague, likely because building Stargate is at this point somewhat theoretical.
No horizon
Qualifier: somewhat
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“What I fear is a cascade effect where artificial intelligence fails to provide businesses with anywhere near enough value to necessitate its costs, which will vastly reduce the amount of consumers and enterprises using or integrating models like ChatGPT, or using their own models on cloud providers like AWS or Azure.”
Our reading: AI may fail to provide enough business value to justify its costs, vastly reducing consumer and enterprise usage of models like ChatGPT.
No horizon
Qualifier: AI fails to provide enough value to justify costs
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“And when you cut through the hype, the AI industry is investing hundreds of billions of dollars to build infrastructure for a future that may never arrive based on the vague promise of hallucination-prone models.”
Our reading: The future that justifies the AI industry's hundreds of billions in infrastructure investment may never arrive.
No horizon
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Conditional forecast
“This scenario is likely if the next generations of ChatGPT or Claude fail to make significant leaps in their capabilities, and as I've said above, their ability to do so is predicated on more training data and compute power than currently exists.”
Our reading: If next-generation ChatGPT or Claude fail to leap in capability, a cascade of reduced AI adoption is likely; such leaps require more data and compute than exists.
No horizon
Only if: if next generations of ChatGPT or Claude fail to make significant capability leaps
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Ironically, OpenAI's best hope for survival would be to fund as many news outlets as possible and directly incentivize them to do in-depth reporting, rather than proliferating a tech that unquestionably harms the media industry.”
Our reading: OpenAI's best hope for survival would be to fund as many news outlets as possible and directly incentivize them to do in-depth reporting, rather than proliferating a technology that harms the media industry.
No horizon
Scope: as many news outlets as possible
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“Running Sora is likely deeply unprofitable, and reducing its generative hallucinations — monkeys with multiple left arms, for example — will require such a remarkable amount of resources that I'm not sure if it'll measurably improve in the next year.”
Our reading: Sora is likely deeply unprofitable and may not measurably improve its hallucinations within the next year.
No horizon
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“Yet under the hood, outside of naked self-dealing, these firms may not be making much money from AI.”
Our reading: Outside self-dealing, Amazon, Microsoft and Google may not be making much money from AI.
No horizon
Qualifier: outside of self-dealing
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“This is, of course, a terrible idea.”
Our reading: Training AI models on synthetic, AI-generated data is a terrible idea.
No horizon
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“As Microsoft doesn't disclose Azure's actual revenue, it's unclear how much money this really is — but 6% of the year-over-year growth isn't exactly exciting.”
Our reading: Because Microsoft doesn't disclose Azure's revenue it is unclear how much money AI contributes, but AI accounting for 6% of Azure's year-over-year growth isn't exactly exciting.
No horizon
Qualifier: as Microsoft doesn't disclose Azure's actual revenue
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“This is why Sam Altman is so desperate to raise trillions to make a new chip company, or get Microsoft to build him a $100 billion supercomputer — because what he's promised isn't possible with today's technology, and may not be possible at all.”
Our reading: What Sam Altman has promised isn't possible with today's technology and may not be possible at all.
No horizon
Source: Bubble Trouble
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“That would be fine if LLMs were kept in the lab, where they arguably still belong”
Our reading: LLMs arguably still belong in the lab rather than in widespread deployment.
No horizon
Qualifier: arguably
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Some get patched, but there very likely will aways be more.”
Our reading: Even as some jailbreaks are patched, there will very likely always be more jailbreak attacks on LLMs.
No horizon
Qualifier: even though some get patched
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“That would be fine if LLMs were kept in the lab, where they arguably still belong, but with hundreds of millions of people using them daoly, and companies like OpenAI promising to put them in all purpose agents with essentially full device-level access, the lack of any kind of guarantee gets more worrisome by the day.”
Our reading: With hundreds of millions of people using LLMs daily and companies like OpenAI promising to put them in all-purpose agents with essentially full device-level access, the lack of any kind of guarantee about LLMs gets more worrisome by the day, though it would be fine if LLMs were kept in the lab, where they arguably still belong.
No horizon
Qualifier: hundreds of millions of people using LLMs daily
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“There seems to be an endless number of ways of attacking them.”
Our reading: There appears to be an endless number of ways to attack LLMs.
No horizon
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“As noted above, Rule 1 of cybersecurity is to keep your attack surface small; in LLMs the attack surface appears to be infinite. That can’t be good.”
Our reading: The attack surface of LLMs appears to be infinite, violating the cybersecurity principle of keeping attack surfaces small.
No horizon
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“If hallucinations were brought down to human expert levels by the end of 2024 I would be truly astonished.”
Our reading: By the end of 2024, hallucinations by large language models will not have been reduced to the level of human experts.
Deadline passed Evidence as of
Limitation: This record does not show how often AI systems or human experts made factual errors in 2024.
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“If OpenAI manages to put out a truly mindblowing GPT-5 this year, the hype will further increase; the bubble won’t burst soon (even if the use cases to justify it aren’t really there).”
Our reading: If OpenAI releases a truly mindblowing GPT-5 in 2024, hype will increase and the GenAI bubble will not burst soon.
Deadline passed Evidence as of
Only if: if OpenAI releases a truly mindblowing GPT-5 this year
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“If nobody (OpenAI, Google, or anyone else) releases a true quantum leap by the end of 2024, substantially addressing key issues around reliability, hallucination, data leakage, and security, the bubble may start to pop by this time next year.”
Our reading: If no one releases a quantum leap addressing reliability, hallucination, data leakage and security by end of 2024, the GenAI bubble may start to pop by March 2025.
No horizon
Only if: if nobody releases a true quantum leap by end of 2024 addressing reliability, hallucination, data leakage and security
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“To be honest, I am not confident Generative AI as currently envisioned will ever come together.”
Our reading: Generative AI as currently envisioned may never come together.
No horizon
Qualifier: as currently envisioned