Claims that can’t be checked by Gary Marcus, page 10 of 24
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Claims that can’t be checked by Gary Marcus (2353)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 10 of 24: claims 901 to 1000 of 2353, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Forecast
“Gone too is a chance to avoid what Senators from Blumenthal to Graham warned about: a repeat of the mess of social media, in which big tech got its way, and society was left paying the consequences.”
Our reading: The chance to avoid AI repeating social media's regulatory failure, where big tech got its way at society's expense, is gone.
No horizon
Source: Two Years Ago Today in AI History: The Tale of An About-face in AI Regulation
Assessment: Can’t be checkedSaid
Forecast
“The cost to humanity of being beguiled by this man may turn out to be enormous.”
Our reading: Being beguiled by Sam Altman may turn out to have an enormous cost to humanity.
No horizon
Source: Two Years Ago Today in AI History: The Tale of An About-face in AI Regulation
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Should we be worried? Hell, yeah.”
Our reading: People should be worried, emphatically so, about powerful people using LLMs to shape their ideas.
No horizon
Source: Has Grok lost its mind and mind-melded with its owner?
Assessment: Can’t be checkedSaid
Observation
“If he couldn’t bend Grok to his will then, he has presumably continued to try.”
Our reading: If Elon Musk could not bend Grok to his will back in February 2025, he has presumably kept trying to do so since then.
No horizon
Only if: if he couldn't bend Grok to his will then
Source: Has Grok lost its mind and mind-melded with its owner?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The federal government should not get to control literally every aspect of how states regulate AI — particularly when they themselves have fallen down on the job —- and the Constitution makes pretty clear that the bill as written is far, far too broad.”
Our reading: The federal government, having fallen down on the job, should not control every aspect of state AI regulation, and the bill is constitutionally far too broad.
No horizon
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“At a time when voters are demanding protection—and global leaders are sounding the alarm—Congress should not tie the hands of the only actors currently positioned to lead.”
Our reading: Congress should not preempt states, the only actors currently positioned to lead on AI regulation.
No horizon
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“This would be deeply problematic under any circumstance, but it’s especially dangerous in the context of a rapidly evolving technology already reshaping healthcare, education, civil rights, and employment.”
Our reading: Banning state AI regulation would be deeply problematic and especially dangerous given AI is already reshaping healthcare, education, civil rights, and employment.
No horizon
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Forecast
“Silicon Valley loves to invoke China to get better terms, but it probably won’t make any difference.”
Our reading: Silicon Valley loves to invoke China to get better terms, but this probably won't make any difference.
No horizon
Qualifier: probably
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If you are among the 82% of Americans who think AI needs oversight, you need to call or write your Congress members now, or the door on AI regulation will slam shut at least for the next decade, if not forever, and we will be entirely at Silicon Valley’s mercy.”
Our reading: Americans who support AI oversight should contact their members of Congress now to oppose the preemption provision.
No horizon
Qualifier: if you support AI oversight
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“A decade of deregulation isn’t a path forward. It’s an abdication of responsibility.”
Our reading: A decade of AI deregulation isn't a path forward; it is an abdication of responsibility.
No horizon
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Forecast
“If Washington wants to pass a comprehensive privacy or AI law with teeth, more power to them, but we all know this is unlikely;”
Our reading: Washington is unlikely to pass a comprehensive privacy or AI law with teeth.
No horizon
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Causal claim
“In insisting that “might makes right” here for AI, they risk setting a dangerous precedent in which whatever party has Federal power makes all the rules, all the time, overriding the power to the States that the 10th Amendment exists to protect, and one of our last remaining checks and balances.”
Our reading: Federal preemption of state AI law risks a dangerous precedent where the party with federal power overrides states' 10th Amendment powers.
No horizon
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Observation
“The provision, which is likely at odds with the 10th Amendment, demands that “no state or political subdivision may enforce any law or regulation regulating artificial intelligence models, artificial intelligence systems, or automated decision systems during the 10-year period beginning on the date of the enactment of this Act.””
Our reading: The proposed 10-year federal preemption of state AI regulation is likely at odds with the 10th Amendment.
No horizon
Source: When it Comes to AI Policy, Congress Shouldn’t Cut States off at the Knees
Assessment: Can’t be checkedSaid
Conditional forecast
“It is a fantasy to think that we can get to safe AI without addressing the problem head on.”
Our reading: Safe AI cannot be achieved unless the reliability/instruction-following problem is addressed directly.
No horizon
Only if: without directly addressing the instruction-following reliability problem
Source: ChatGPT Blows Mapmaking 101
Assessment: Can’t be checkedSaid
Causal claim
“Gary Marcus is certain that neurosymbolic AI can help.”
Our reading: Neurosymbolic AI can help address the reliability problems of current models.
No horizon Evidence as of
Limitation: No evidence was used in this assessment. The two sources considered were not relied on, and the result does not depend on their findings.
Source: ChatGPT Blows Mapmaking 101
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The most important question about AI isn’t a technical question.”
Our reading: The most important question about AI is not a technical question.
No horizon
Source: The Pope gets it
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The most important question is about how to maintain and grow a just society in the age of AI.”
Our reading: The most important question about AI is how to maintain and grow a just society in the age of AI.
No horizon
Source: The Pope gets it
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We are going to do need a more thorough going redesign than that.”
Our reading: Education will need a redesign more thorough than one-off assignments like having students find hallucinations in LLM-generated papers.
No horizon
Source: ”Everyone is cheating their way through college” with GenAI. Who should bear the costs?
Assessment: Can’t be checkedSaid
Forecast
“The likely outcome here? College students from here on out will get little from their educations, professors will be overwhelmed, unable to adapt to the scale of what would be required to do individualized education with budgets that don’t really support that, and the Sam Altmans of the world will laugh their way to the bank.”
Our reading: The likely outcome is that from here on out college students will get little from their educations, professors will be overwhelmed and unable to adapt to the scale required for individualized education on budgets that don't really support it, and the Sam Altmans of the world will laugh their way to the bank.
No horizon
Qualifier: with budgets that don't really support individualized education
Source: ”Everyone is cheating their way through college” with GenAI. Who should bear the costs?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The curriculum does, obviously need a redesign.”
Our reading: University curricula need to be redesigned because of GenAI.
No horizon
Qualifier: in light of GenAI cheating
Source: ”Everyone is cheating their way through college” with GenAI. Who should bear the costs?
Assessment: Can’t be checkedSaid
Observation
“The status quo clearly no longer works.”
Our reading: The current university education model no longer works now that GenAI exists.
No horizon
Qualifier: given GenAI
Source: ”Everyone is cheating their way through college” with GenAI. Who should bear the costs?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“You can’t cut funding to universities, signal to kids that using ChatGPT to write the papers is ok, and expect professors to save the day and the educational system without financial support.”
Our reading: One cannot cut university funding, signal to students that using ChatGPT to write their papers is acceptable, and still expect professors to save the day and the educational system without financial support.
No horizon
Scope: funding to universities is cut
Source: ”Everyone is cheating their way through college” with GenAI. Who should bear the costs?
Assessment: Can’t be checkedSaid
Forecast
“This morning I woke up to not one but two reports that make clear that the era of Big Brother is about to enter the United States in a massive new ways, and not just via OpenAI but through multiple vectors:”
Our reading: Two reports make clear that the era of Big Brother is about to enter the United States in massive new ways, not just via OpenAI but through multiple vectors.
No horizon
Qualifier: not just via OpenAI but through multiple vectors
Source: All your data belong to us
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I couldn’t agree more. If you are not worried, you are not paying attention. The time to speak up is now.”
Our reading: People should speak up now against the growing AI-driven surveillance.
No horizon
Source: All your data belong to us
Assessment: Can’t be checkedSaid
Forecast
“I had started warning that bad things were coming last year when I realized that OpenAI’s only financially move might be head into surveillance, and started noticing some of the steps they were taking in that direction, like putting Paul Nakasone (former head the NSA) on their board last June, and taking a share in a webcam company last August.”
Our reading: OpenAI's only financially viable path may be to move into surveillance, and it has already taken steps in that direction such as adding Paul Nakasone to its board and investing in a webcam company.
No horizon
Qualifier: as its only financially viable move
Source: All your data belong to us
Assessment: Can’t be checkedSaid
Causal claim
“Things have taken a severe turn for the worse, and it’s not just that it was reported last month that OpenAI now trying to launch a social media network, presumably in part towards the end of collecting further private data.”
Our reading: OpenAI's reported effort to launch a social media network is presumably motivated in part by a wish to collect more private data.
No horizon
Qualifier: presumably
Source: All your data belong to us
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Even the best performance (15% hallucination rate) is, relative to an open-book exam with sources supplied, pathetic.”
Our reading: Even the best performance in the cited study, a 15% hallucination rate, is pathetic for an open-book exam where the sources were supplied.
No horizon
Scope: cited benchmark of 60 questions with CNN source articles supplied
Assessment: Can’t be checkedSaid
Observation
“LLMs have their uses – next-token prediction is great for a kind of fancy autocomplete for coding, for example — but it is a fallacy to think that because GenAI outputs sound human-like that their computations are human-like.”
Our reading: LLMs are useful e.g. for coding autocomplete, but their human-sounding outputs do not imply human-like computation.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“But LLMs still cannot - and on their own may never be able to — reliably do even things that basic.”
Our reading: LLMs on their own may never be able to reliably check their output against basic sources.
No horizon
Qualifier: on their own
Assessment: Can’t be checkedSaid
Observation
“LLMs mimic the rough structure of human language, but 8 years and roughly half a trillion dollars after their introduction, they continue to lack a grasp of a reality.”
Our reading: After 8 years and roughly half a trillion dollars, LLMs still lack a grasp of reality.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We will eventually get to something better, but continuing to put all our eggs in the Henrietta’s LLM basket is absurd to the point of being delusional.”
Our reading: The field should not put all its eggs in the LLM basket; doing so is delusional.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But attempting to use the graph to make predictions about the capacities of future AI is misguided, and the fact that it went viral is a sign that in AI (as in so many other domains) people are repeating things that they want to believe, based on their alleged conclusions rather than their validity.”
Our reading: Using the METR graph to predict future AI capabilities is misguided, and its virality reflects people repeating what they want to believe.
No horizon
Source: The latest AI scaling graph - and why it hardly makes sense
Assessment: Can’t be checkedSaid
Observation
“That giant leap seems especially unwarranted given that there has likely been a lot of recent data augmentation directed towards software benchmarks in particular (where this is feasible).”
Our reading: There has likely been a lot of recent data augmentation directed at software benchmarks in particular (where feasible), which makes the leap from software results to tasks in general seem especially unwarranted.
No horizon
Scope: where this is feasible
Source: The latest AI scaling graph - and why it hardly makes sense
Assessment: Can’t be checkedSaid
Observation
“Search and replace LaMDA with Claude, and it all still applies.”
Our reading: Marcus's 2022 argument that LaMDA was not sentient and lacked understanding applies equally to Anthropic's Claude.
No horizon
Source: New adventures in AI hype: “Our language models are so ‘conscious’ we need to give them rights”
Assessment: Can’t be checkedSaid
Causal claim
“I suspect the real move here is simply, as it so often is, to hype the product — basically by saying, hey, look at how smart our product is, it’s so smart we need to give it rights.”
Our reading: Anthropic's work on evaluating the 'welfare' of its models is probably mainly a commercial move to hype its product as smart enough to need rights.
No horizon
Source: New adventures in AI hype: “Our language models are so ‘conscious’ we need to give them rights”
Assessment: Can’t be checkedSaid
Forecast
“None of the software in either of those systems has survived in modern efforts at “artificial general intelligence”, and I am not sure that LaMDA and its cousins will play any important role in the future of AI, either.”
Our reading: LaMDA and similar language models may not play any important role in the future of AI.
No horizon
Qualifier: 2022 argument reaffirmed as applying to today's LLMs
Source: New adventures in AI hype: “Our language models are so ‘conscious’ we need to give them rights”
Assessment: Can’t be checkedSaid
Observation
“I still don’t remotely see an argument that current models are sentient, nor any argument that scaling a model makes it more conscious, even if it can mimic more language from humans discussing consciousness.”
Our reading: Marcus says he still does not remotely see an argument that current models are sentient, nor any argument that scaling a model makes it more conscious, even if a scaled model can mimic more human language about consciousness.
No horizon
Qualifier: even if it can mimic more language from humans discussing consciousness
Source: New adventures in AI hype: “Our language models are so ‘conscious’ we need to give them rights”
Assessment: Can’t be checkedSaid
Rhetoric
“And given that Meta also appears to have just juiced some benchmarks, the whole thing is starting to look like a bunch of over-promisers scrambling to make things look better than they really are.”
Our reading: Given that Meta also appears to have just juiced some benchmarks, the whole thing is starting to look like a bunch of over-promisers scrambling to make things look better than they really are.
No horizon
Qualifier: Meta appears to have just juiced some benchmarks
Source: OpenAI’s dirty December o3 demo doesn’t readily replicate
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The truth is that we don’t really know how good o3 is or isn’t, and nobody should ever take OpenAI’s video presentations particularly seriously again, until they have been fully vetted by the community.”
Our reading: We don't really know how good o3 is or isn't, and nobody should ever again take OpenAI's video presentations particularly seriously until the community has fully vetted them.
No horizon
Qualifier: until they have been fully vetted by the community
Source: OpenAI’s dirty December o3 demo doesn’t readily replicate
Assessment: Can’t be checkedSaid
Rhetoric
“Forgive if me I am getting Theranos vibes.”
Our reading: Marcus says he is getting Theranos vibes from OpenAI's handling of the o3 results.
No horizon
Qualifier: vibes
Source: OpenAI’s dirty December o3 demo doesn’t readily replicate
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“some possible explanations are given, but this is again a very bad look.”
Our reading: The gap between OpenAI's reported 32% and Epoch's 11% for o3-mini on FrontierMath is again a very bad look for OpenAI, even though some possible explanations have been given.
No horizon
Qualifier: some possible explanations are given
Source: OpenAI’s dirty December o3 demo doesn’t readily replicate
Assessment: Can’t be checkedSaid
Observation
“And the lower 56% is not much different from what Jacob Andreas’s lab at MIT got in November. It’s arguably worse; if I followed correctly, and if the measures are the same, Andreas lab’s best score was actually higher, at 61%.”
Our reading: The 56% is not much different from what Jacob Andreas's lab at MIT got in November, and arguably worse, since, if Marcus followed correctly and the measures are the same, the Andreas lab's best score was higher at 61%.
No horizon Evidence as of
Limitation: This disposition concerns the qualified comparison, not the existence of the November result.
Source: OpenAI’s dirty December o3 demo doesn’t readily replicate
Assessment: Can’t be checkedSaid
Observation
“His claims about o3 yesterday didn’t survive even the day.”
Our reading: Tyler Cowen's claim that OpenAI's o3 is AGI was refuted within a day of its release.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“The real reason for doing this of course is to get more data; and as I have pointed out before, to pivot OpenAI into surveillance.”
Our reading: The real reason for Sam Altman's move to cut into Elon Musk's turf is to get more data and to pivot OpenAI into surveillance.
No horizon
Scope: regarding Sam Altman's move to cut into Elon Musk's turf
Source: Altman vs Musk cage match
Assessment: Can’t be checkedSaid
Causal claim
“The real reason for doing this of course is to get more data; and as I have pointed out before, to pivot OpenAI into surveillance.”
Our reading: OpenAI's move into Musk's turf is part of an effort to pivot OpenAI into surveillance.
No horizon
Qualifier: regarding OpenAI's move into Musk's (social network) turf
Source: Altman vs Musk cage match
Assessment: Can’t be checkedSaid
Observation
“But the big change we need still hasn’t come.”
Our reading: As of April 2025, the paradigm change AI needs has not happened.
No horizon
Source: ”Those claiming we’re mere months away from AI agents replacing most programmers” should think again
Assessment: Can’t be checkedSaid
Conditional forecast
“The only way we are going to get past this kind of 80:20, sometimes it works, sometimes it doesn’t kind of AI is to change the paradigm.”
Our reading: AI will not get past its 80:20, sometimes-works reliability unless the paradigm changes.
No horizon
Only if: without a change of paradigm
Source: ”Those claiming we’re mere months away from AI agents replacing most programmers” should think again
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“As I have argued here before, Sam Altman is, despite outward appearances, not a nice fellow.”
Our reading: Sam Altman, despite outward appearances, is not a nice person.
No horizon
Qualifier: despite outward appearances
Source: Sam Altman’s attitude problem
Assessment: Can’t be checkedSaid
Forecast
“With “GPT-5” level models remaining elusive, this picture may not change enormously in the near-term.”
Our reading: With GPT-5-level models still elusive, the poor ROI picture for GenAI may not change enormously in the near term.
No horizon
Qualifier: GPT-5 level models remaining elusive
Source: Poor ROI for GenAI
Assessment: Can’t be checkedSaid
Observation
“With “GPT-5” level models remaining elusive, this picture may not change enormously in the near-term.”
Our reading: As of April 2025, GPT-5-level models have not yet arrived.
No horizon Evidence as of
Limitation: The April 4 announcement alone does not establish release status on April 10, 2025.
Source: Poor ROI for GenAI
Assessment: Can’t be checkedSaid
Causal claim
“One reason may be that it is starting to become clear that GenAI is not delivering the ROI (return on investment) people expected.”
Our reading: NVidia's sharp stock decline may be partly because it is becoming clear that GenAI is not delivering the expected return on investment.
No horizon
Qualifier: as a possible reason for NVidia's decline
Source: Poor ROI for GenAI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Media, get out there and find out what happened.”
Our reading: The media should investigate why AB-501 was gutted.
No horizon
Assessment: Can’t be checkedSaid
Rhetoric
“Just because Meta or OpenAI wants you to believe something doesn’t mean it is true.”
Our reading: Meta or OpenAI wanting people to believe something does not by itself make it true.
No horizon
Source: Deep Learning, Deep Scandal
Assessment: Can’t be checkedSaid
Observation
“The reality, reported or otherwise, is that large language models are no longer living up to expectations, and its purveyors appear to be making dodgy choices to keep that fact from becoming obvious.”
Our reading: LLMs are no longer living up to expectations, and their vendors appear to be making dodgy choices to conceal this.
No horizon
Source: Deep Learning, Deep Scandal
Assessment: Can’t be checkedSaid
Observation
“OpenAI failed at this (calling their best result GPT-4.5, and recently announcing a further delay on GPT-5); Grok failed at this (Grok 3 is no GPT 5). Google has failed at reaching “GPT-5” level, Anthropic has, too.”
Our reading: As of the post's publication on April 7, 2025, OpenAI, xAI (Grok), Google and Anthropic had each failed to reach what Marcus calls "GPT-5 level" AI.
No horizon
Qualifier: as of April 7, 2025
Source: Deep Learning, Deep Scandal
Assessment: Can’t be checkedSaid
Observation
“One of the clearest signs of this is the saga of the just-released Llama 4, the latest failed billion (?) dollar attempt by one of the majors to create what we might call GPT-5 level AI.”
Our reading: Meta's Llama 4 is a failed attempt to create GPT-5 level AI.
No horizon Evidence as of
Limitation: This assessment does not establish whether Llama 4 met internal targets or expectations.
Source: Deep Learning, Deep Scandal
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“LLMs are not the way. We definitely need something better.”
Our reading: LLMs are not the right path for AI, and better alternative approaches are needed.
No horizon
Source: Scaling is over, the bubble may be deflating, LLMs still can’t reason, and you can’t trust Sam
Assessment: Can’t be checkedSaid
Observation
“Generative AI may indeed be turning out to be a dud, financially.”
Our reading: Generative AI may be turning out to be a financial dud.
No horizon
Source: Scaling is over, the bubble may be deflating, LLMs still can’t reason, and you can’t trust Sam
Assessment: Can’t be checkedSaid
Observation
“Every one of my 25 predictions for 2025 still seems to be on track.”
Our reading: As of April 2025, all 25 of Marcus's predictions for 2025 seem to be on track.
No horizon
Qualifier: seems
Source: Scaling is over, the bubble may be deflating, LLMs still can’t reason, and you can’t trust Sam
Assessment: Can’t be checkedSaid
Observation
“the financial bubble may be bursting”
Our reading: The AI financial bubble may be bursting as of April 2025.
No horizon
Source: Scaling is over, the bubble may be deflating, LLMs still can’t reason, and you can’t trust Sam
Assessment: Can’t be checkedSaid
Observation
“And the bubble might possibly finally be deflating.”
Our reading: The generative AI bubble might possibly be deflating as of April 2025.
No horizon
Source: Scaling is over, the bubble may be deflating, LLMs still can’t reason, and you can’t trust Sam
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Sam Altman can’t be trusted”
Our reading: Sam Altman cannot be trusted.
No horizon
Source: Scaling is over, the bubble may be deflating, LLMs still can’t reason, and you can’t trust Sam
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Moreover, however you slice it, this need of the AIs for special cajoling before they will think carefully about questions that are presented to them is a serious weakness, and the demand for special prompting tilts the scale.”
Our reading: However you slice it, the AIs' need for special cajoling before they will think carefully about questions presented to them is a serious weakness, and the demand for special prompting tilts the scale.
No horizon
Qualifier: however you slice it
Source: Reports of LLMs mastering math have been greatly exaggerated
Assessment: Can’t be checkedSaid
Forecast
“It is by no means certain that additional user prompts will add much,.”
Our reading: Additional user prompting may not substantially improve reasoning LLMs' performance on such problems.
No horizon
Qualifier: models already designed for complex reasoning
Source: Reports of LLMs mastering math have been greatly exaggerated
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The really important challenge is not to get the AIs to solve more USAMO problems; it is to get them to say “I give up” when they can’t.”
Our reading: The really important challenge is not getting AIs to solve more USAMO problems but getting them to say 'I give up' when they can't solve them.
No horizon
Source: Reports of LLMs mastering math have been greatly exaggerated
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The refusal of these kinds of AI to admit ignorance or incapacity and their obstinate preference for generating incorrect but plausible-looking answers instead are one of their most dangerous characteristics.”
Our reading: The refusal of these kinds of AI (LLMs) to admit ignorance or incapacity, and their obstinate preference for generating incorrect but plausible-looking answers instead, are among their most dangerous characteristics.
No horizon
Scope: these kinds of AI (LLMs like those tested on the USAMO)
Source: Reports of LLMs mastering math have been greatly exaggerated
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Getting AIs to answer “I don’t know” is one of the most important unsolved challenges facing the field.”
Our reading: Getting AIs to answer 'I don't know' is one of the most important unsolved challenges facing the field.
No horizon
Source: Reports of LLMs mastering math have been greatly exaggerated
Assessment: Can’t be checkedSaid
Conditional forecast
“If this kind of technology becomes commonly used to answer difficult questions before the problem of generating invalid answers is fixed, we will be in serious trouble.”
Our reading: If LLMs become commonly used for difficult questions before their invalid-answer problem is fixed, society will be in serious trouble.
No horizon
Only if: LLMs become commonly used to answer difficult questions
Source: Reports of LLMs mastering math have been greatly exaggerated
Assessment: Can’t be checkedSaid
Observation
“And, wait for it, the whole thing was probably recommended to them by an LLM:”
Our reading: The Trump administration's tariff-rate formula was probably recommended to it by an LLM.
No horizon
Assessment: Can’t be checkedSaid
Observation
“The near-universal consensus is that President Trump’s new tariff policy (quite possibly partly written by LLMs) is likely to be among the worst economic policies in history.”
Our reading: Trump's April 2025 tariff policy was quite possibly written partly by LLMs.
No horizon
Qualifier: quite possibly
Source: Questions about President Trump from a former psychology professor
Assessment: Can’t be checkedSaid
Conditional forecast
“In a faceoff, though, it might actually beat ChatGPT.”
Our reading: In a faceoff, Eugene Goostman might actually beat ChatGPT.
No horizon
Only if: in a faceoff
Source: AI has (sort of) passed the Turing Test; here’s why that hardly matters
Assessment: Can’t be checkedSaid
Observation
“Because, as I (and many others) have said for years, the Turing Test is a test of human gullibility, not a test of intelligence.”
Our reading: The Turing Test is a test of human gullibility, not a test of intelligence.
No horizon
Qualifier: as he and many others have said for years
Source: AI has (sort of) passed the Turing Test; here’s why that hardly matters
Assessment: Can’t be checkedSaid
Observation
“Fooling people for a few minutes is not the measure of intelligence, and never was.”
Our reading: Fooling people for a few minutes is not the measure of intelligence, and never was.
No horizon
Source: AI has (sort of) passed the Turing Test; here’s why that hardly matters
Assessment: Can’t be checkedSaid
Observation
“None of these systems, in my judgment, are anywhere near as intelligent as naive audiences might think.”
Our reading: None of the Turing Test chatbots, from ELIZA to ChatGPT 4.5, are anywhere near as intelligent as naive audiences might think.
No horizon
Source: AI has (sort of) passed the Turing Test; here’s why that hardly matters
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Few people in the AI community take the Turing Test seriously, and few people should.”
Our reading: Few people in the AI community take the Turing Test seriously, and few should.
No horizon
Source: AI has (sort of) passed the Turing Test; here’s why that hardly matters
Assessment: Can’t be checkedSaid
Conditional forecast
“If the CoreWeave IPO turns out to be a debacle, people may also sour on NVidia, too. Things could get wild, fast.”
Our reading: If the CoreWeave IPO is a debacle, people may sour on NVidia too, and things could get wild quickly.
No horizon
Only if: if the CoreWeave IPO turns out to be a debacle
Source: Unwoven
Assessment: Can’t be checkedSaid
Conditional forecast
“If it does go badly, there may well be shockwaves, perhaps affecting the whole AI industry.”
Our reading: If CoreWeave's stock market debut goes badly, there may be shockwaves possibly affecting the whole AI industry.
No horizon
Only if: if CoreWeave's NASDAQ debut goes badly
Source: Unwoven
Assessment: Can’t be checkedSaid
Observation
“But the economics do not and have not ever made sense, relative to the realities of the techonology.”
Our reading: The economics of generative AI do not make sense and never have, given the technology's actual capabilities.
No horizon
Qualifier: relative to the realities of the technology
Assessment: Can’t be checkedSaid
Forecast
“I genuinely believe arguments from Stuart Russell and others that AI could someday be a trillion dollar annual market.”
Our reading: AI could someday be a trillion-dollar annual market.
No horizon
Qualifier: someday
Assessment: Can’t be checkedSaid
Rhetoric
“But so much seems to have been built on dreams.”
Our reading: So much in GenAI seems to have been built on dreams.
No horizon
Qualifier: seems
Assessment: Can’t be checkedSaid
Forecast
“Different, yet-to-be-developed approaches, with a firmer connection to the world of symbolic AI (perhaps hybrid neurosymbolic models) might well prove to be vastly more valuable.”
Our reading: New approaches more connected to symbolic AI, perhaps neurosymbolic hybrids, might prove vastly more valuable than GenAI.
No horizon
Qualifier: yet-to-be-developed approaches
Assessment: Can’t be checkedSaid
Forecast
“But the use of GenAI will likely continued to be stymied by hallucinations and silly errors.”
Our reading: Use of generative AI will likely continue to be held back by hallucinations and silly errors.
No horizon
Assessment: Can’t be checkedSaid
Observation
“There are a bunch of reasons to think that the deflation of the bubble may have already started.”
Our reading: The GenAI bubble may already have begun to deflate as of March 2025.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“But unlocking that market will require something new: a different kind of AI that is reliable and trustworthy.”
Our reading: Reaching a trillion-dollar AI market will require a new, reliable and trustworthy kind of AI beyond GenAI.
No deadline stated Evidence as of
Only if: unlocking the trillion-dollar market
Assessment: Can’t be checkedSaid
Forecast
“Companies like OpenAI may to some degree supplement revenue by selling very private user data to intelligence agencies and the like, to be sure.”
Our reading: Companies like OpenAI may partly supplement revenue by selling private user data to intelligence agencies and similar buyers.
No horizon
Qualifier: to some degree
Assessment: Can’t be checkedSaid
Forecast
“LLMs may ultimately prove to be something like a $$30B or $40B industry, which is not nothing — but they could well also turn out to be an industry that costs double or treble that to run, including data licensing, high-priced staff, massively expensive data centers that may become obsolete quickly, and seemingly endless litigation.”
Our reading: The LLM industry may ultimately be worth around $30-40B in revenue while costing two to three times that to run.
No horizon
Qualifier: ultimately
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Anthropic needs to answer some hard questions here.”
Our reading: Anthropic should answer hard questions about the gap between its claims about AI coding and its hiring of coders.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“I stressed, for example, there would likely be an ongoing need to have humans to debug, maintain and systems.”
Our reading: There would likely be an ongoing need for humans to debug and maintain systems.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“As I explained, Amodei’s coding claims seemed outright fanciful to me.”
Our reading: Amodei's coding claims seemed outright fanciful to Marcus.
No horizon
Qualifier: seemed ... to me
Assessment: Can’t be checkedSaid
Observation
“My guess is that current AI coding tools would struggle to use 6502 well, for multiple reasons, including a lack of high-level libraries, fewer Stack Overflow queries to rely on, and a greater needed for the coding tool to understanding high level architectural concerns that might be side-stepped in higher level languages.”
Our reading: Marcus guesses that current AI coding tools would struggle to use 6502 assembly well, for multiple reasons including a lack of high-level libraries, fewer Stack Overflow queries to rely on, and a greater need to understand high-level architectural concerns that might be side-stepped in higher-level languages.
No horizon
Qualifier: my guess
Source: AI Coding Fantasy meets Pac-Man
Assessment: Can’t be checkedSaid
Forecast
“I think most or all of the points I made – about challenges in building and debugging novel systems with sufficient generality- will stand the test of time.”
Our reading: Marcus thinks that most or all of the points he made last week, about the challenges of building and debugging novel systems with sufficient generality, will stand the test of time.
No horizon
Qualifier: I think
Source: AI Coding Fantasy meets Pac-Man
Assessment: Can’t be checkedSaid
Rhetoric
“But I also fear a world in which the State department can judge anyone, at any time, to be a “threat” to the state, even based on superficial AI analysis, and deport them without due process, using AI as a smoke screen to authoritarian action.”
Our reading: Marcus fears the State Department could use superficial AI analysis to label anyone a threat and deport them without due process, with AI serving as a smoke screen for authoritarian action.
No horizon
Source: Urgent warning: Black Mirror has entered the United States, with AI as its handmaiden
Assessment: Can’t be checkedSaid
Observation
“Nobel-calibre science is a lot harder than driving, which we still have not reliably solved”
Our reading: Nobel-calibre science is much harder than driving, and AI still has not reliably solved driving.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Per the 80:20 rule I described a few days ago, it’s easy to make demos; 80% of driving, 80% of some scientific problem, etc. Getting the last 20% is often really really hard.”
Our reading: In AI it is easy to build demos covering 80% of a task such as driving or a scientific problem, but getting the last 20% is often very hard.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“A set of problems that have resisted solution for a quarter century, even backed by a half trillion dollars, probably aren’t all going to get solved so quickly.”
Our reading: The long-standing core AGI problems probably will not all be solved quickly, even with half a trillion dollars of funding.
No horizon
Qualifier: even backed by a half trillion dollars
Source: Ezra Klein’s new take on AGI – and why I think it’s probably wrong
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If it is this unreliable on arithmetic, it will be this unreliable on everything else, and we shouldn’t accord it the society-changing power that people intend to accord it.”
Our reading: An AI system this unreliable on arithmetic will be similarly unreliable on everything else and should not be given society-changing power.
No horizon
Qualifier: if it is this unreliable on arithmetic
Source: Ezra Klein’s new take on AGI – and why I think it’s probably wrong
Assessment: Can’t be checkedSaid
Conditional forecast
“The fact that GPT 4.5 is wildly expensive relative to its competitors, and the fact that future models following the same trajectory of scale might be prohibitively expensive, especially in a regime in which nobody has a moat and price wars have been intensifying since the release of DeepSeek.”
Our reading: Future models that keep scaling the same way might be prohibitively expensive, especially since nobody has a moat and price wars are intensifying.
No horizon
Only if: if future models follow the same trajectory of scale
Source: Ezra Klein’s new take on AGI – and why I think it’s probably wrong
Assessment: Can’t be checkedSaid
Conditional forecast
“I completely agree with Klein that AGI will matter a lot when it comes.”
Our reading: When AGI arrives, it will matter a lot.
No horizon
Only if: when AGI arrives
Source: Ezra Klein’s new take on AGI – and why I think it’s probably wrong
Assessment: Can’t be checkedSaid
Observation
“It actually seems to me that US may be all but surrendering the AGI race over to China.”
Our reading: With its academic research institutions being dismantled, the US may be all but surrendering the AGI race to China.
No horizon
Qualifier: given the dismantling of academic research institutions
Source: Ezra Klein’s new take on AGI – and why I think it’s probably wrong
Assessment: Can’t be checkedSaid
Forecast
“Having observed the pace of science and engineering firsthand over the last half century, the middle scenario seems to me like – by far – the most plausible.”
Our reading: The most plausible scenario by far is that getting to AGI will be a slog: some problems get solved in the next 2-3 years and others take longer.
No horizon
Source: Ezra Klein’s new take on AGI – and why I think it’s probably wrong