Claims checked against evidence, page 12 of 15
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims checked against evidence (1432)
Each of these claims was checked against dated evidence.
Page 12 of 15: claims 1101 to 1200 of 1432, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: SupportedSaid
Gary Marcus Observation
“Just like we can’t guarantee that they won’t make stuff up, we don’t know how to guarantee that they won’t write flawed code.”
Our reading: We cannot guarantee that AI systems will not fabricate or write flawed code.
No horizon Evidence as of
Limitation: This is an empirical assessment of contemporary general-purpose systems, not a mathematical impossibility claim.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: UnresolvedSaid
Gary Marcus Forecast
“And emphasize that wholesale, automated fake news will be used both to manipulate markets and elections, and might undermine democracy.”
Our reading: Wholesale, automated fake news will be used to manipulate both markets and elections, and it might undermine democracy.
No deadline stated Evidence as of
Limitation: Wholesale and automated require interpretation.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: UnresolvedSaid
Gary Marcus Observation
“Your best example of a spectacular machine invention is in fact a perfect example of the kind of parroting and theft of intellectual property that is characteristic of large language models.”
Our reading: Bard's 'baby shoes' story was parroting of an existing story, and such parroting and IP theft is characteristic of large language models.
No horizon Evidence as of
Limitation: The available transcript is an updated version, and the independent transcription contains only part of the output.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: SupportedSaid
Gary Marcus Observation
“When it comes to AI in particular, we are getting way ahead of our skis, rushing forward a technology we don’t fully understand.”
Our reading: Humanity is rushing forward with AI, a technology it does not fully understand.
No horizon Evidence as of
Limitation: Support applies to deployment proceeding despite incomplete understanding.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: SupportedSaid
Gary Marcus Observation
“There’s some information in there, but whatever’s there is often both patchy and problematic.”
Our reading: Chatbots contain some information, but whatever is there is often both patchy and problematic.
No horizon Evidence as of
Limitation: Biography and family-relationship tasks are not representative of every chatbot use.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: SupportedSaid
Gary Marcus Observation
“The connections in chatbots are very different from the connections in the brain; it’s a mistake to compare them directly in this way.”
Our reading: The connections in chatbots are very different from those in the brain, so comparing them directly in the way Hinton did (by counting connections) is a mistake.
No horizon Evidence as of
Limitation: The evidence does not establish which system learns better on a specified task.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: SupportedSaid
Gary Marcus Observation
“Actually … a lot of us still think you are declaring victory prematurely.”
Our reading: A lot of people, Marcus included, still think Hinton is declaring victory for neural networks prematurely.
No horizon Evidence as of
Limitation: The result depends on reading 'premature victory' as unresolved scientific problems rather than an explicit response to Hinton.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: UnresolvedSaid
Gary Marcus Observation
“You know how many radiologists have been replaced by machines seven years later? Zero.”
Our reading: Seven years after Hinton's 2016 prediction, no radiologists have been replaced by machines.
No horizon Evidence as of
Limitation: The workforce census covers the UK, while the claim supplies no geographic limit.
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“I can’t either; there’s a lot we can do to help, but nothing I can see either to absolutely guarantee safety. Rushing ahead is creating risk.”
Our reading: A lot can be done to help with AI safety, but Marcus can see nothing that would absolutely guarantee it, and rushing ahead is creating risk.
No horizon Evidence as of
Limitation: The company reports do not isolate the effect of development or deployment speed [e1, e2].
Source: What was 60 Minutes thinking, in that interview with Geoff Hinton?
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“There is no internal model of what she actually did, and hence no capacity to support reliable inference, hence no guarantee that anything it writes about her will be true.”
Our reading: LinkedIn's OpenAI-powered assistant has no internal model of what Kaya Yurieff actually did, and hence no capacity to support reliable inference about her, and hence no guarantee that anything it writes about her will be true.
No horizon Evidence as of
Limitation: No evidence identifying the version, inputs, prompts or complete outputs of the assistant tested by Yurieff.
Source: Muddles about Models
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“Without flexible underlying models that can reliably support inferences on novel questions, we can expect more of the same of what we are seeing over and over now: more hallucinations, more reasoning that is hit or miss, and nothing like the stability we should insist on for future AI.”
Our reading: Without flexible underlying models, AI systems will keep producing hallucinations and hit-or-miss reasoning, and will lack the needed stability.
No deadline stated Evidence as of
Only if: without flexible underlying models that reliably support inference on novel questions
Source: Muddles about Models
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“It’s the same problem: correlations are fragile, and you can’t count on them.”
Our reading: GPT's failure to stick to the rules of chess, even when it plays a decent game, reflects the same problem: correlations are fragile and cannot be counted on.
No horizon Evidence as of
Limitation: A failure to follow rules does not by itself reveal the model’s internal representations.
Source: Muddles about Models
Assessment: UnresolvedSaid
Gary Marcus Observation
“By my count, though I acknowledge I may have missed one, there are at least 7 falsehoods.”
Our reading: A recent Microsoft Bing answer about the House Speaker contained at least seven falsehoods.
No horizon Evidence as of
Limitation: The underlying political facts are more accessible than the original model output.
Source: Seven lies in four sentences
Assessment: SupportedSaid
Gary Marcus Observation
“Rumors that LLMs have solved then hallucination problem are greatly exaggerated.”
Our reading: LLMs have not solved the hallucination problem.
No horizon Evidence as of
Limitation: The research was a preprint at publication.
Source: Seven lies in four sentences
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“This all happens not because LLMs are smart, in the sense of being able to compare what they say with the world, but simply because they are good at making new sentences that sound old sentences, the essence of autocomplete.”
Our reading: LLMs produce plausible falsehoods because they mimic existing sentences like autocomplete rather than checking what they say against the world.
No horizon Evidence as of
Limitation: Training descriptions establish design facts, not a complete causal explanation of errors.
Source: Seven lies in four sentences
Assessment: SupportedSaid
Gary Marcus Observation
“Explicit verbal instruction is key in training human employees, and LLMs aren’t very good at following those instructions.”
Our reading: LLMs are not very good at following explicit verbal instructions.
No horizon Evidence as of
Limitation: The conclusion depends on reading the claim as a reliability criticism.
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: SupportedSaid
Gary Marcus Observation
“Hallucinations are a huge, unsolved issue.”
Our reading: LLM hallucinations are a huge, unsolved problem.
No horizon Evidence as of
Limitation: Huge is a qualitative judgment; no universal hallucination rate is established.
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: SupportedSaid
Gary Marcus Forecast
“The competitors may not (yet) be as good, but they are free, which will suit some customers fine.”
Our reading: Free open-source competitors to OpenAI, though perhaps not yet as good, will suit some customers.
Settled before its deadline Evidence as of
Limitation: The deployment and reception account comes from a Meta case study containing customer testimony, without an independent evaluation.
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: UnresolvedSaid
Gary Marcus Observation
“This symmetry fail is mighty big, a mighty persistent error that has endured for decades.”
Our reading: The symmetry failure is a big, persistent error in neural networks that has endured for decades.
No horizon Evidence as of
Limitation: Failure to locate a matching historical experiment is not evidence that none exists.
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: SupportedSaid
Gary Marcus Observation
“On made-up facts, in a first experiment, the model was at zero percent correct, on celebrities, in a second experiment, performance was still dismal.”
Our reading: In the Berglund et al. experiments, the model scored zero percent on reversed made-up facts and performed dismally on reversed celebrity facts.
No horizon Evidence as of
Limitation: The other synthetic reversal direction scored 0.1%, so zero should not be generalized to every test.
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: SupportedSaid
Gary Marcus Observation
“Forty one years later, neural networks (at least of the popular variety) still struggle with this. They still remain pointillistic masses of blurry memory, never as systematic as reasoning machines ought to be.”
Our reading: Popular neural networks still struggle with systematic relational reasoning and are not as systematic as reasoning machines should be.
No horizon Evidence as of
Limitation: The tested models and tasks do not establish that every neural architecture lacks systematic reasoning [e1].
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: SupportedSaid
Gary Marcus Observation
“The shocking (yet, in historical context, see below, unsurprising) answer is no:”
Our reading: LLMs trained on 'A is B' cannot automatically infer that 'B is A'.
No horizon Evidence as of
Limitation: These experiments do not establish impossibility across all models, training procedures or prompting methods [e1].
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: SupportedSaid
Gary Marcus Observation
“Voluntary guidelines aren’t laws, and they aren’t enforcement mechanisms, either; companies can still do as they please.”
Our reading: At least in the United States, voluntary AI guidelines are not laws or enforcement mechanisms, and companies can still do as they please.
No horizon Evidence as of
Limitation: The result depends on reading the sentence in its immediate White House-commitments context.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“If we don’t know what is in the data sets, we can’t know what biases these systems will spread through society if they are widely used, we can’t know the extent to which individual content creatore are being exploited, and worse any outside efforts to mitigate risks are greatly undermined.”
Our reading: If we don't know what is in AI training data sets, we can't know what biases these systems will spread through society if widely used or the extent to which individual content creators are being exploited, and outside efforts to mitigate risks are greatly undermined.
No horizon Evidence as of
Only if: if we don't know what is in the data sets
Assessment: SupportedSaid
Gary Marcus Observation
“At least in the United States, policy around AI is still largely notional, not actual.”
Our reading: US AI policy remains largely notional rather than actual.
No horizon Evidence as of
Limitation: The result depends on the surrounding discussion of new commercial AI safeguards.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“The apparent economics of AI (we will see if they yet come to pass as imagined) are too seductive to too many, to people building AI companies, to people investing in AI companies, to people getting paid 6 and even 7 digit salaries, and to people expecting campaign donations from big tech companies.”
Our reading: The apparent economics of AI, which may or may not come to pass as imagined, are too seductive to too many people (AI company builders, investors, people paid 6- and 7-digit salaries, and those expecting big-tech campaign donations) for any AI moratorium to happen in the near term.
No horizon Evidence as of
Limitation: Public statements may omit motives or present them selectively.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“We are obviously never going to have a moratorium anytime in the near-term.”
Our reading: There will be no moratorium on AI development in the near term.
No deadline stated Evidence as of
Limitation: The main contemporaneous account comes from the organization advocating the pause.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“We have vastly more attention on AI policy, throughout the world, and I definitely give the pause letter (which I signed but didn’t write) some credit for that.”
Our reading: Global attention on AI policy has vastly increased since March 2023, and the pause letter deserves some credit for that.
No horizon Evidence as of
Limitation: A new meeting is not a worldwide measure of attention.
Assessment: SupportedSaid
Gary Marcus Observation
“Large language models still make stuff up (“hallucinate”), still can’t be trusted in their reasoning, and remain unreliable.”
Our reading: LLMs still hallucinate and their reasoning cannot be trusted.
No horizon Evidence as of
Limitation: The results concern specific models, tasks, and prompts rather than every possible use.
Assessment: SupportedSaid
Gary Marcus Forecast
“And even if this specific example is patched up, there will inevitably be others of a similar flavor.”
Our reading: Even if this specific LLM error is patched, other errors of a similar flavor will inevitably appear.
Settled before its deadline Evidence as of
Limitation: The result depends on the stated readings of 'even if', 'similar flavor', and 'inevitably'.
Source: Doug Lenat, 1950-2023
Assessment: UnresolvedSaid
Gary Marcus Observation
“That has meant, for example, no tangible progress on getting the big companies to commit to revealing what data their models were trained on; we have transparency in name, but not in deed.”
Our reading: There has been no tangible progress in getting big AI companies to commit to disclosing their training data.
No horizon Evidence as of
Limitation: The result depends on distinguishing existing disclosures from commitments to disclose.
Source: Could we all be doomed, without ranked-choice voting?
Assessment: SupportedSaid
Gary Marcus Observation
“Indeed, what we have seen on the AI front so far in the US, for the most part, is voluntary guidelines – stuff that the companies are comfortable with.”
Our reading: US AI policy so far has mostly been voluntary guidelines the companies are comfortable with.
No horizon Evidence as of
Limitation: This is a qualitative assessment of prominent federal initiatives, not a count of every federal, state, or local measure.
Source: Could we all be doomed, without ranked-choice voting?
Assessment: UnresolvedSaid
Gary Marcus Forecast
“The default right now is regulatory capture: the companies will make the rules, telling the governments what they are and are not comfortable with. The government will go along.”
Our reading: By default, AI companies will write the rules for AI regulation and governments will go along.
No deadline stated Evidence as of
Limitation: The evidence is not a census of US AI regulation or industry lobbying.
Source: Could we all be doomed, without ranked-choice voting?
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“Without data transparency, we can’t make sure our models are without bias, or even know the sources of the biases they hide within, we can’t truly understand the scope and limits of their generalization (which is central to understanding what they might do in unexpected circumstances) and we are hamstrung in our ability to mitigate their harmful consequences: what is unknown is hard to fix.”
Our reading: Without transparency about the data models were trained on, we can't make sure models are free of bias or even know the sources of their biases, we can't truly understand the scope and limits of their generalization, and we are hamstrung in our ability to mitigate their harmful consequences.
No horizon Evidence as of
Limitation: The supporting paper synthesizes several forms of access; their effects are not interchangeable.
Source: Could we all be doomed, without ranked-choice voting?
Assessment: SupportedSaid
Gary Marcus Observation
“When I think about how poor his company (Meta) is even at making LLMs that don’t hallucinate, I get worried.”
Our reading: Meta is poor at making LLMs that do not hallucinate.
No horizon Evidence as of
Limitation: TruthfulQA measures a challenging question set, not typical user traffic.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: UnresolvedSaid
Gary Marcus Observation
“But if and when we get there, will they be dominating? Non-dominating? The honest answer is we haven’t a clue.”
Our reading: Marcus says that, honestly, we haven't a clue whether super-smart machines, if and when we get there, will be dominating or non-dominating.
No horizon Evidence as of
Limitation: The search cannot establish a universal absence of applicable research.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: UnresolvedSaid
Gary Marcus Forecast
“We aren’t going to be able calculate p(doom) to three decimal places.”
Our reading: No one will be able to calculate p(doom) precisely, to three decimal places.
No deadline stated Evidence as of
Limitation: Search coverage is not proof that a qualifying method does not exist.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: UnresolvedSaid
Gary Marcus Observation
“Yet a really large fraction of the AI and AI-adjacent community doesn’t really want to talk about any of this at all.”
Our reading: A large fraction of the AI and AI-adjacent community does not want to discuss AI doom or catastrophe risk at all.
No horizon Evidence as of
Limitation: The packet contains no representative measurement of willingness to discuss these risks during the target period.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: SupportedSaid
Gary Marcus Observation
“this too seems like a rather large assumption that is far from proven, a possibility but not demonstrably a default,”
Our reading: Yudkowsky's claim that AGI will kill everyone by default is far from proven; it is possible but not shown to be the default.
No horizon Evidence as of
Limitation: The conclusion involves evaluating the sufficiency of arguments, not checking a numerical measurement.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: UnresolvedSaid
Gary Marcus Observation
“Some of the specific arguments Yudkowsky and Bostrom have made may well be weak, but there is no solid argument that the number is zero, or even that the risk is infinitesimal.”
Our reading: There is no solid argument that AI doom risk is zero or even infinitesimal, though some arguments by Yudkowsky and Bostrom may be weak.
No horizon Evidence as of
Limitation: A broad negative claim about all arguments cannot be established by sampling a few positions.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: SupportedSaid
Gary Marcus Observation
“What would make artificial general intelligence general is generality: the ability to cope with new things. And that’s precisely where current AI is still lacking.”
Our reading: Generality, the ability to cope with new things, is what would make AGI general, and that is precisely where current AI is still lacking.
No horizon Evidence as of
Limitation: The outcome depends on reading 'lacking' as substantial limitations, rather than complete absence.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“This is even more true four years later, now that we have systems that ingest massive amounts of data from both public and commercial sources, quite possibly including eg sample law school and medical exams etc.”
Our reading: Because current systems ingest massive data possibly including sample exams, benchmark success is even less indicative of intelligence than in 2019.
No horizon Evidence as of
Limitation: OpenAI's audit was a developer assessment with acknowledged detection limits.
Assessment: SupportedSaid
Gary Marcus Observation
“But it is also well-known throughout the industry that benchmarks are often quite gameable.”
Our reading: AI benchmarks are often quite gameable.
No horizon Evidence as of
Limitation: The evidence does not estimate the proportion of all benchmarks affected or measure industry-wide awareness [e1, e2].
Assessment: UnresolvedSaid
Gary Marcus Observation
“Without a sense of what it is in the training corpus, it’s almost impossible to conclude anything about what a machine truly understands and how general that is.”
Our reading: Without knowing the training corpus, one can hardly conclude what a machine understands or how general it is.
No horizon Evidence as of
Limitation: A reading limited to whether any bounded behavioral conclusion is possible would weigh more strongly against Marcus.
Assessment: SupportedSaid
Gary Marcus Observation
“Basically, beating benchmarks doesn’t tell you as much as you might think. Often when you do, it’s a statistical artifact.”
Our reading: Beating AI benchmarks is often a statistical artifact and says less than people assume.
No horizon Evidence as of
Limitation: No representative estimate establishes how often this happens across all AI benchmarks.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Humans can write summaries of things without hallucination; LLMs cannot do so reliably.”
Our reading: LLMs cannot reliably write summaries without hallucinating.
No horizon Evidence as of
Limitation: The inferred monthly observation window is narrower than a general review of 2023 models.
Assessment: UnresolvedSaid
Gary Marcus Observation
“18 months ago, before the LLM euphoria, we all took for granted that AGI wasn’t just about beating ordinary humans.”
Our reading: Around early 2022, before LLM euphoria, the field assumed AGI meant more than beating ordinary humans.
No horizon Evidence as of
Limitation: The later review provides historical definitions, not a representative measurement of February 2022 views [e1].
Assessment: UnresolvedSaid
Gary Marcus Observation
“in my view it’s a subtle redefinition that dramatically moves the goal posts.”
Our reading: In Marcus's view, Legg's definition of AGI as better than typical human performance on virtually all cognitive tasks is a subtle redefinition that dramatically moves the goal posts, compared with how Legg and Marcus publicly discussed the term last year.
No horizon Evidence as of
Limitation: Missing original exchange records prevent a finding either way.
Assessment: SupportedSaid
Gary Marcus Observation
“The dirty little secret that pretty much everyone in the AI community knows is that doing well on a benchmark is not actually tantamount to being intelligent.”
Our reading: Doing well on a benchmark does not amount to being intelligent.
No horizon Evidence as of
Limitation: The assessment depends on reading intelligent as broad, reliable competence in the surrounding AGI discussion.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Nobody has yet figured out how to build a driverless car that can deal with that fact.”
Our reading: Nobody has yet figured out how to build a driverless car that can deal with the real world's endless crazy and out-of-the-ordinary circumstances.
No horizon Evidence as of
Limitation: Reliable handling and acceptable human assistance are not defined in the source.
Source: Face it, self-driving cars still haven’t earned their stripes
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“Edge cases are everywhere; they are in driving, they are in medicine, they there will be a zillion of them humanoid robots, if and when we roll them out. Anyone who thinks any of this is going to be easy is fooling themselves.”
Our reading: Edge cases are everywhere, including in driving and medicine; there will be a zillion of them in humanoid robots if and when they are rolled out; and anyone who thinks any of this will be easy is fooling themselves.
No deadline stated Evidence as of
Only if: if and when humanoid robots are rolled out
Source: Face it, self-driving cars still haven’t earned their stripes
Assessment: UnresolvedSaid
Gary Marcus Observation
“In fact, the truth is we don’t actually have any truly self-driving cars yet.”
Our reading: There are not yet any truly self-driving cars: every 'self-driving vehicle' on public roads has either a human safety driver on board or a remote human who can help it when issues arise.
No horizon Evidence as of
Limitation: The strongest contemporaneous evidence covers the two San Francisco operators.
Source: Face it, self-driving cars still haven’t earned their stripes
Assessment: UnresolvedSaid
Gary Marcus Observation
“We have no such thing; building it would require a major reshaping of research directions, towards an AI based in facts and competent in reasoning, rather than just the powerful tools for text prediction that we have now.”
Our reading: We have no AI smart enough to know when it is being used to spew falsehood, and building it would require a major reshaping of research directions toward AI based in facts and competent in reasoning, rather than just the powerful text-prediction tools we have now.
No horizon Evidence as of
Limitation: The cited evaluations precede the publication-date snapshot and are context.
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: UnresolvedSaid
Gary Marcus Observation
“It’s a powerful reminder that our existing laws are not sufficient to protect us from the full scope of harms that generative AI is beginning to enable .”
Our reading: Existing laws are not sufficient to protect against the full scope of harms generative AI is beginning to enable.
No horizon Evidence as of
Limitation: The legal evidence primarily concerns federal election advertising.
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: SupportedSaid
Gary Marcus Causal claim
“It highlights the inadequacy of the voluntary agreements such as the one that 7 tech companies recently made with The White House. Those agreements called for watermarking, but because they are voluntary, they do nothing to prevent bad actors from rolling their own automated misinformation software.”
Our reading: Because the White House AI agreements with 7 tech companies are voluntary, they are inadequate and do nothing to stop bad actors building their own automated misinformation software.
No horizon Evidence as of
Limitation: The result depends on reading do nothing as no direct constraint on independent development.
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Automatic tools like these will surely accelerate the enshittification of the internet that I discussed yesterday.”
Our reading: Automatic misinformation generation tools will accelerate the degradation of the internet.
No deadline stated Evidence as of
Limitation: The assessment uses misinformation contamination as the ordinary contextual meaning of degradation.
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: SupportedSaid
Gary Marcus Forecast
“Social media companies better watch out, too; they will be polluted in much the same way as the internet as a whole.”
Our reading: Social media platforms will be polluted by automated misinformation much as the internet as a whole will be.
Settled before its deadline Evidence as of
Limitation: Support depends on reading the comparison as a shared mechanism, not equal prevalence.
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: SupportedSaid
Gary Marcus Observation
“And it’s on: the LLM-driven enshittification of the internet.”
Our reading: LLM-driven degradation (enshittification) of the internet is already under way.
No horizon Evidence as of
Limitation: AI attribution was generally based on content evidence rather than access to generation records.
Assessment: SupportedSaid
Gary Marcus Observation
“And of course LLMs are fully capable of make up truly terrible advice, too, like this disconcerting recipe from last week:”
Our reading: LLMs are fully capable of making up truly terrible advice, such as a disconcerting recipe from the previous week.
No horizon Evidence as of
Limitation: The example used deliberately hazardous ingredients; it does not establish how often harmful advice occurs in ordinary use [e1].
Assessment: UnresolvedSaid
Gary Marcus Observation
“And even worse, Amazon is starting to be a receptacle for inaccurate and exploitative quickie histories of recent events:”
Our reading: Amazon is starting to be a receptacle for inaccurate and exploitative quickie histories of recent events.
No horizon Evidence as of
Limitation: The retained dated evidence does not complete verification of specific material errors in the book.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Coding is a solid use case that will continue to keep generative AI in the limelight for a long time. Programmers will never go back.”
Our reading: Coding will remain a solid use case keeping generative AI prominent for a long time, and programmers will not abandon it.
No deadline stated Evidence as of
Limitation: Survey findings describe respondents, not a census of programmers.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Driverless cars are powered by AI, but so far the money that has gone into them, on order of $100 billion dollars, vastly exceeds the actual revenue that has come out of them.”
Our reading: Investment in driverless cars, around $100 billion, vastly exceeds the revenue they have generated so far.
No horizon Evidence as of
Limitation: Corroborating the investment magnitude alone does not establish the complete claim.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Perhaps the only use case that really seemed compelling to either of us economically was search (e.g., using Bing powered by ChatGPT instead of Google Search) — but the technical problems there are immense; there is no reason to think that the hallucination problem will be solved soon.”
Our reading: There is no reason to think that the hallucination problem, part of the immense technical problems facing the search use case, will be solved soon.
No horizon Evidence as of
Limitation: The two studies do not comprehensively assess the leading approaches or their prospects for near-term deployment.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“I think coders will remain happy with generative AI assistance; its autocomplete nature is fabulous for their line of work, and they have the training to detect and fix the not-infrequent errors.”
Our reading: Marcus thinks coders will remain happy with generative AI assistance, because its autocomplete nature is fabulous for their work and they have the training to detect and fix its not-infrequent errors.
No deadline stated Evidence as of
Limitation: Survey respondents are not necessarily representative of all programmers.
Assessment: SupportedSaid
Gary Marcus Observation
“We have not one, but many, serious, unsolved problems at the core of generative AI — ranging from their tendency to confabulate (hallucinate) false information, to their inability to reliably interface with external tools like Wolfram Alpha, to the instability from month to month (which makes them poor candidates for engineering use in larger systems).”
Our reading: Generative AI has many serious unsolved core problems, including hallucination, unreliable tool use, and month-to-month instability.
No horizon Evidence as of
Limitation: API-Bank does not directly evaluate Wolfram Alpha or every model available in August 2023.
Assessment: SupportedSaid
Gary Marcus Observation
““Scaling” systems by making them larger has helped in some ways, but not others; we still really cannot guarantee that any given system will be honest, harmless, or helpful, rather than sycophantic, dishonest, toxic or biased.”
Our reading: Scaling has helped only partially; no system can yet be guaranteed honest, harmless, or helpful.
No horizon Evidence as of
Limitation: The scaling results concern selected tasks and model series; they do not show that scaling generally fails.
Assessment: UnresolvedSaid
Gary Marcus Observation
“And, reality check, we have no concrete reason, other than sheer technoptimism, for thinking that solutions to any of these problems is imminent.”
Our reading: There is no concrete reason, other than sheer techno-optimism, for thinking that solutions to any of generative AI's core problems (such as hallucination, unreliable tool use and month-to-month instability) are imminent.
No horizon Evidence as of
Limitation: This is an assessment of contemporary grounds for expectation, not a forecast scored against later events.
Assessment: SupportedSaid
Gary Marcus Observation
“The technology we have now is built on autocompletion, not factuality.”
Our reading: Current generative AI is built on autocompletion rather than factuality.
No horizon Evidence as of
Limitation: The outcome depends on reading built on as the foundation rather than every component of a deployed system.
Assessment: SupportedSaid
Gary Marcus Observation
“We also know, for example, that Bing has defamed people, and it has misread articles as saying the opposite of what they actually say, in service of doing so.”
Our reading: Bing has defamed people, and in doing so has misread articles as saying the opposite of what they actually say.
No horizon Evidence as of
Limitation: One connected incident is established; no prevalence or victim count is inferred.
Source: Two models of AI oversight - and how things could go deeply wrong
Assessment: UnresolvedSaid
Gary Marcus Observation
“I don’t see any reason to think these plugins are in fact safe. (They are far less sandboxed and less rigorously controlled than Apple app store apps.)”
Our reading: Marcus says he does not see any reason to think ChatGPT plugins are in fact safe, noting that they are far less sandboxed and less rigorously controlled than Apple App Store apps.
No horizon Evidence as of
Limitation: The packet lacks a sufficiently complete account of OpenAI's review and enforcement during the target period.
Source: Two models of AI oversight - and how things could go deeply wrong
Assessment: SupportedSaid
Gary Marcus Observation
“Meanwhile, ChatGPT plugins (produced by OpenAI, which they have a close tie with) open a wide range of security problems: those plugins can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all alarms to any security professional.”
Our reading: ChatGPT plugins open a wide range of security problems, since they can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all of which would alarm any security professional.
No horizon Evidence as of
Limitation: Support concerns the ecosystem's capabilities and attack mechanisms, not every individual plugin.
Source: Two models of AI oversight - and how things could go deeply wrong
Assessment: UnresolvedSaid
Gary Marcus Observation
“Neurosymbolic AI might offer a better way of resolving these issues, since it traffics in facts and explicit reasoning, but neurosymbolic AI is out of fashion, and neither well-developed nor strongly supported by the VC world.”
Our reading: Neurosymbolic AI is out of fashion, not well developed, and not strongly supported by venture capital.
No horizon Evidence as of
Limitation: The field boundaries and all three qualitative thresholds are unspecified.
Source: How China might crush the West in the race to AGI if we don’t up our regulatory game
Assessment: UnresolvedSaid
Gary Marcus Observation
“At present, however, there is no legal requirement in the west that AI companies resolve the truthfulness or alignment issues.”
Our reading: There is currently no legal requirement in the West for AI companies to resolve truthfulness or alignment issues.
No horizon Evidence as of
Limitation: The schema has no source type for official regulatory announcements; official_statistics is used for the government documents.
Source: How China might crush the West in the race to AGI if we don’t up our regulatory game
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Their forte has been and always will be making shit up; they can’t for the life of them (speaking metaphorically of course) check facts.”
Our reading: For GPT models on their own, making things up has been and always will be their forte, and they cannot check facts.
No deadline stated Evidence as of
Limitation: A literal zero-ability reading would yield a different assessment of the fact-checking component.
Source: GPT-5 and irrational exuberance
Assessment: ContradictedSaid
Gary Marcus Observation
“Essentially none of the fancy cognitive things GPT-4 is rumored to do stand up.”
Our reading: Essentially none of the advanced cognitive abilities attributed to GPT-4 hold up under scrutiny.
No horizon Evidence as of
Limitation: The result depends on counting controlled reasoning performance among the unspecified 'fancy cognitive things'.
Source: GPT-5 and irrational exuberance
Assessment: SupportedSaid
Gary Marcus Observation
“Maarten Sap from Yejin Choi’s lab dove deeper, with a more thorough investigation, and confirmed what we suspected: the claim that large language models have mastered theory of mind is a myth:”
Our reading: The claim that large language models have mastered theory of mind is a myth.
No horizon Evidence as of
Limitation: The studies test particular models, tasks and prompts, rather than every aspect of theory of mind [e5, e6].
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Forecast
“It will be even better than GPT-4 at creating convincing sounding prose.”
Our reading: GPT-5 will be better than GPT-4 at creating convincing-sounding prose.
No deadline stated Evidence as of
Limitation: Writing preference varies with genre, prompting, and generation settings.
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Forecast
“GPT-5 is not going to be magic. It will not give China, the US, or anyone else interstellar starships; it will not give anyone an insuperable military advantage.”
Our reading: GPT-5 will not give China, the US, or anyone else interstellar starships, and it will not give anyone an insuperable military advantage.
No deadline stated Evidence as of
Limitation: Military capabilities may not be publicly observable.
Source: GPT-5 and irrational exuberance
Assessment: SupportedSaid
Gary Marcus Observation
“It’s not smart enough to filter out falsehood, but just smart enough to be dangerous, creating and spreading falsehoods it fails to verify.”
Our reading: GPT cannot filter out falsehood and creates and spreads falsehoods it fails to verify, making it dangerous.
No horizon Evidence as of
Limitation: Volokh's unsuccessful searches support his assessment that the examples were fabricated, but are not exhaustive proof that no corresponding source exists [e2].
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Observation
“(Not one of my predictions about GPT-4 proved to be incorrect; every flaw I predicted would persist persisted.)”
Our reading: Marcus says none of his predictions about GPT-4 proved incorrect, and every flaw he predicted would persist in GPT-4 did persist.
No horizon Evidence as of
Limitation: Confirming several central predictions cannot establish a universal track-record claim.
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Observation
“GPT-4 can’t even play a good game of chess (it’s about as good as chess computers from 1978).”
Our reading: GPT-4 cannot play a good game of chess and is about as good as 1978 chess computers.
No horizon Evidence as of
Limitation: The admitted GPT-4 evidence consists of one recorded game and an account of two attempted games [e1, e2].
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Observation
“And very little of what GPT-4 does is fully replicable. Sometimes summarization (aka compression) works (super cool!) and sometimes it doesn’t, as many were quick to point out:”
Our reading: Very little of what GPT-4 does is fully replicable; its summarization sometimes works and sometimes fails.
No horizon Evidence as of
Limitation: The task population and meaning of full replication are unspecified.
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“Some form of AI may eventually do everything people are imagining, revolutionizing science and technology and so on, but LLMs will be at most only a tiny part of whatever as-yet-uninvented technology does that.”
Our reading: If some as-yet-uninvented form of AI eventually does everything people are imagining, such as revolutionizing science and technology, LLMs will be at most only a tiny part of it.
No deadline stated Evidence as of
Only if: if some form of AI eventually does everything people are imagining
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Forecast
“It might be on the nice-to-have list; it will not mark the mythical singularity.”
Our reading: GPT-5 will not mark the singularity.
No deadline stated Evidence as of
Limitation: The cited failure rate concerns one task, not a measurement of sustained, self-amplifying intelligence improvement [e1].
Source: GPT-5 and irrational exuberance
Assessment: UnresolvedSaid
Gary Marcus Observation
“Vulnerable patients shouldn’t be talking to chatbots that aren’t competent for this situation, and too much of the discussion in the media have made chatbots sound more intelligent and even more friendly than they are.”
Our reading: Too much media discussion has made chatbots sound more intelligent and friendly than they are.
No horizon Evidence as of
Limitation: The phrase “too much” remains partly evaluative.
Assessment: SupportedSaid
Gary Marcus Observation
“The fundamental issue is that chatbots are neither reliable nor truthful, and at the same time don’t really grasp things like human psychology.)”
Our reading: Chatbots are neither reliable nor truthful and do not really grasp human psychology.
No horizon Evidence as of
Limitation: The result depends on the operational reading of really grasp.
Assessment: SupportedSaid
Gary Marcus Forecast
“Two reasons, first, because I believe that more incidents like this will happen, perhaps many more, hundreds or thousands, given how rapidly chatbots are being deployed, and the near total lack of regulation on how they can be used.”
Our reading: More chatbot-associated incidents like this suicide will happen, perhaps hundreds or thousands, given rapid deployment and near-total lack of regulation.
Settled before its deadline Evidence as of
Limitation: The filings do not establish causation or liability [e1, e2, e3].
Assessment: MixedSaid
Gary Marcus Observation
“Two reasons, first, because I believe that more incidents like this will happen, perhaps many more, hundreds or thousands, given how rapidly chatbots are being deployed, and the near total lack of regulation on how they can be used.”
Our reading: There is a near-total lack of regulation on how chatbots can be used, and they are being deployed rapidly.
No horizon Evidence as of
Limitation: The result depends on reading regulation to include existing binding law.
Assessment: SupportedSaid
Gary Marcus Observation
“The Chatbot involved was GPT-J, not GPT-3 or ChatGPT (contra some news headlines that were erroneous).”
Our reading: The chatbot involved in the Belgian suicide case was GPT-J, not GPT-3 or ChatGPT.
No horizon Evidence as of
Limitation: The evidence is reporting based on developer interviews, rather than an independently inspected deployment record.
Assessment: SupportedSaid
Gary Marcus Observation
“Whether or not you want a pause, the reality is that large language models remain exactly what I have always described them to be: bulls in a china shop, powerful, reckless, and difficult to control.”
Our reading: Large language models remain powerful, reckless, and difficult to control.
No horizon Evidence as of
Limitation: Reckless is assessed as a description of outputs, not intent.
Assessment: SupportedSaid
Gary Marcus Causal claim
“At some level the petition worked; Capitol Hill, for example, pretty clearly got the message that we can’t sit around forever on these issues.”
Our reading: At some level the FLI pause petition worked; for example, Capitol Hill pretty clearly got the message that we can't sit around forever on AI issues.
No horizon Evidence as of
Limitation: The evidence supports a limited claim of congressional attention and favorable response, not Congress-wide agreement [e1].
Assessment: UnresolvedSaid
Gary Marcus Observation
“Virtually everyone, even at OpenAI, has acknowledged that there are serious risks, but thus far few tangible steps have been taken to mitigate them—either by government or industry.”
Our reading: Although virtually everyone, including people at OpenAI, acknowledges serious AI risks, government and industry have so far taken few tangible steps to mitigate them.
No horizon Evidence as of
Limitation: Government guidance, legal requirements and technical safeguards differ substantially in reach and effectiveness.
Source: The Open Letter Controversy
Assessment: SupportedSaid
Gary Marcus Observation
“it didn't call for a ban on the vast majority of AI research, only for a brief pause on one very specific project with a technology that has *known* risks with no known solutions.”
Our reading: The one very specific project the letter asked to pause, training systems exceeding GPT-4, involves a technology that has known risks with no known solutions.
No horizon Evidence as of
Limitation: The outcome depends on interpreting solutions as dependable resolution rather than any risk reduction.
Source: The Open Letter Controversy
Assessment: ContradictedSaid
Gary Marcus Observation
“None of the top labs actually is remotely transparent, governance basically doesn’t exist yet, and there are basically no safeguards actually in place.”
Our reading: None of the top AI labs is remotely transparent.
No horizon Evidence as of
Limitation: The result depends on counting substantive model disclosures as meaningful laboratory transparency.
Source: The Open Letter Controversy
Assessment: SupportedSaid
Gary Marcus Observation
“I personally haven't changed; I still think that LLMs are unreliable, and still think that they are a very poor basis for factuality.”
Our reading: LLMs are unreliable.
No horizon Evidence as of
Limitation: The evidence is developer disclosure rather than an independent cross-model audit.
Source: The Open Letter Controversy
Assessment: SupportedSaid
Gary Marcus Observation
“A company called Adept.AI just raised $350 million dollars to do just that, to allow large language models to access, well, pretty much everything (aiming to “supercharge your capabilities on any software tool or API in the world” with LLMs, despite their clear tendencies towards hallucination and unreliability).”
Our reading: Adept.AI just raised $350 million to let large language models access pretty much everything, despite LLMs' clear tendencies toward hallucination and unreliability.
No horizon Evidence as of
Limitation: Adept's main domain blocked the page reader; the quotations come from readable reproductions on adept.futurefabric.co.
Source: AI risk ≠ AGI risk
Assessment: SupportedSaid
Gary Marcus Causal claim
“But here’s the thing: although a lot of the literature equates artificial intelligence risk with the risk of superintelligence or artificial general intelligence, you don’t have to be superintelligent to create serious problems.”
Our reading: AI does not need to be superintelligent to create serious problems.
No horizon Evidence as of
Limitation: This incident establishes that serious harm is possible, not how frequently it occurs [e1].
Source: AI risk ≠ AGI risk
Assessment: SupportedSaid
Gary Marcus Observation
“The problem of hallucinations is not solved; reliability is not solved; planning on complex tasks is (as the authors themselves acknowledge) not solved.”
Our reading: GPT-4 has not solved hallucinations, reliability, or planning on complex tasks.
No horizon Evidence as of
Limitation: The planning example concerns the early research model examined in Sparks.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“The paper’s core claim— “GPT-4 attains a form of general intelligence [as] demonstrated by its core mental capabilities (such as reasoning, creativity, and deduction)”—literally cannot be tested with serious scrutiny, because the scientific community has no access to the training data.”
Our reading: The Sparks of AGI paper's claim that GPT-4 attains a form of general intelligence cannot be seriously tested because scientists have no access to its training data.
No horizon Evidence as of
Limitation: This is not support for the claim that all serious testing requires training-data access.
Assessment: UnresolvedSaid
Gary Marcus Observation
“if the definition of reasoning is to obtain valid conclusions from known facts, GPT-4 frequently falls short there, too.”
Our reading: GPT-4 frequently fails to reason, if reasoning means reaching valid conclusions from known facts.
No horizon Evidence as of
Limitation: Later evaluations were not rejected merely because they were published after the post; the problem is their uncertain connection to the target model period.
Source: This Week in AI Doublespeak