Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Changes, page 22 of 37

Every substantive change to the register, newest first. Speaker events record what the people covered said or changed. Site events record what this site published, corrected, flagged, or withdrew. The two never share a label.

Substantive changes (7383)

Page 22 of 37: changes 4201 to 4400 of 7383, newest first. Dates are when the change was recorded.

  1. Assessment publishedBy this siteGary Marcus

    Because current systems ingest massive data possibly including sample exams, benchmark success is even less indicative of intelligence than in 2019.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-virginia-agi-is-not-imminent-27976b-a1.

  2. Assessment publishedBy this siteGary Marcus

    Marcus seriously doubts that AI is close to beating typical human performance on virtually all cognitive tasks.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-virginia-agi-is-not-imminent-26c457-a1.

  3. Assessment publishedBy this siteGary Marcus

    Generality, the ability to cope with new things, is what would make AGI general, and that is precisely where current AI is still lacking.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-virginia-agi-is-not-imminent-15eaa7-a1.

  4. HeldBy this siteGary Marcus

    Held record marcus-no-virginia-agi-is-not-imminent-09b64e

    The openai screen doubts the attribution.

  5. Assessment publishedBy this siteGary Marcus

    AGI standards should not be lowered so big tech can prematurely declare victory, since much work remains.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-virginia-agi-is-not-imminent-04db25-a1.

  6. HeldBy this siteGary Marcus

    Held record marcus-no-sora-has-not-learned-physics-404d76

    The anthropic screen reads the claim differently.

  8. HeldBy this siteGary Marcus

    Held record marcus-no-rag-is-probably-not-going-to-rescue-f16f8e

    The anthropic screen reads the claim differently.

  9. Assessment publishedBy this siteGary Marcus

    RAG is immature, not guaranteed correct, and there is no evidence it is a cure-all for LLM problems.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-ee4809-a2.

  10. Assessment publishedBy this siteGary Marcus

    Getting RAG from a research idea to reliable production remains difficult.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-e0f14c-a1.

  12. Assessment publishedBy this siteGary Marcus

    RAG may somewhat reduce hallucinations but will not come close to eliminating them.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-afdf2d-a2.

  13. Assessment publishedBy this siteGary Marcus

    Every LLM, with or without RAG, should probably carry a disclaimer after each response.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-a483c2-a1.

  15. Assessment publishedBy this siteGary Marcus

    So far retrieval-augmented generation has produced inconsistent results.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-38ee57-a1.

  16. Assessment publishedBy this siteGary Marcus

    Investors have put over $100 billion into driverless cars and so far have little to show for it.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-28a2dc-a1.

  17. Assessment publishedBy this siteGary Marcus

    Expecting RAG to fix LLMs' problems is like waiting for a miracle.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-282d32-a1.

  21. Assessment publishedBy this siteGary Marcus

    It seems likely that Sam Altman will return to OpenAI and the board will resign.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-one-person-should-be-trusted-here-895f9e-a1.

  23. Assessment publishedBy this siteGary Marcus

    OpenAI's original structure was meant to make the for-profit serve the non-profit, with a financially disinterested board looking out for humanity.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-one-person-should-be-trusted-here-5d3e24-a1.

  24. HeldBy this siteGary Marcus

    Held record marcus-no-one-person-should-be-trusted-here-49c08d

    The openai screen reads the claim differently.

  25. Assessment publishedBy this siteGary Marcus

    Reaching AGI will require new ideas, not merely new code optimizations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-need-to-panic-about-anthropics-fd88df-a1.

  26. HeldBy this siteGary Marcus

    Held record marcus-no-need-to-panic-about-anthropics-d9b71e

    The final-text check failed: rationale_supported_by_evidence, citations_correct; overclaims: The report retains human responsibility for choosing research goals..

  27. HeldBy this siteGary Marcus

    Held record marcus-no-need-to-panic-about-anthropics-cece9a

    The final-text check failed: citations_correct.

  28. Assessment publishedBy this siteGary Marcus

    Pure deep learning has largely hit a wall, and neurosymbolic AI is rescuing it.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-need-to-panic-about-anthropics-beac4c-a1.

  29. Assessment publishedBy this siteGary Marcus

    AGI, a machine that can autonomously do anything a human can do, has not been achieved.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-need-to-panic-about-anthropics-bcf02b-a1.

  30. Assessment publishedBy this siteGary Marcus

    Achieving AGI is harder than achieving recursive self-improvement.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-need-to-panic-about-anthropics-8e85fb-a1.

  31. Assessment publishedBy this siteGary Marcus

    Anthropic's blog shows considerable coding advances, and its argument for slowing down AI research should be considered.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-need-to-panic-about-anthropics-89bf48-a1.

  32. Assessment publishedBy this siteGary Marcus

    There is no need to panic yet about Anthropic's results.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-need-to-panic-about-anthropics-3672ac-a1.

  33. Assessment publishedBy this siteGary Marcus

    A faster AI coding tool will probably not end the world.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-need-to-panic-about-anthropics-353225-a1.

  34. Assessment publishedBy this siteGary Marcus

    Users of popular GenAI systems might accidentally get into legal trouble because the systems do not list sources or warn about possible infringement.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-multimodal-chatgpt-is-not-going-e402ee-a1.

  35. HeldBy this siteGary Marcus

    Held record marcus-no-multimodal-chatgpt-is-not-going-e04f38

    The openai review found the criteria unfair to the speaker's words.

  36. HeldBy this siteGary Marcus

    Held record marcus-no-multimodal-chatgpt-is-not-going-c7862e

    "likely" is hedged or probabilistic; one outcome does not score it; hedged language is not scored, even on a plain reading

  38. Assessment publishedBy this siteGary Marcus

    GenAI developers will not easily escape copyright problems, and copyright infringement lawyers will have plenty of work.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-multimodal-chatgpt-is-not-going-bc5816-a1.

  39. Assessment publishedBy this siteGary Marcus

    Almost nothing in AI is truly trivial.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-multimodal-chatgpt-is-not-going-b5a5d4-a1.

  41. HeldBy this siteGary Marcus

    Held record marcus-no-multimodal-chatgpt-is-not-going-8bc786

    The openai review abstained.

  42. Assessment publishedBy this siteGary Marcus

    Gary Marcus still thinks that licensing is the best and only way to go in dealing with generative AI's copyright problems.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-multimodal-chatgpt-is-not-going-79fee3-a1.

  43. HeldBy this siteGary Marcus

    Held record marcus-no-multimodal-chatgpt-is-not-going-63ace3

    The anthropic review found the criteria unfair to the speaker's words.

  44. Assessment publishedBy this siteGary Marcus

    In a 45-minute test, multimodal ChatGPT was unreliable at identifying the copyrighted sources of Midjourney-generated images.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-multimodal-chatgpt-is-not-going-5a9763-a2.

  45. Assessment publishedBy this siteGary Marcus

    Output similarity may be central in many upcoming lawsuits, and their outcome could greatly affect the economics of generative AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-multimodal-chatgpt-is-not-going-13aa21-a1.

  47. HeldBy this siteGary Marcus

    Held record marcus-no-dario-amodei-we-will-not-be-curing-ebd3ce

    The anthropic screen doubts the attribution.

  48. Assessment publishedBy this siteGary Marcus

    AI will not make the disease-curing contributions Hassabis described within the next decade, though it might someday.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-dario-amodei-we-will-not-be-curing-d2d3a2-a1.

  49. HeldBy this siteGary Marcus

    Held record marcus-no-dario-amodei-we-will-not-be-curing-aacab9

    The anthropic screen reads the claim differently.

  50. HeldBy this siteGary Marcus

    Held record marcus-no-dario-amodei-we-will-not-be-curing-9c5a21

    The anthropic screen reads the claim differently.

  53. Assessment publishedBy this siteGary Marcus

    If Amodei believes his claims, he should do a medical internship to better understand the challenges in medicine.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-dario-amodei-we-will-not-be-curing-3451f4-a1.

  54. HeldBy this siteGary Marcus

    Held record marcus-no-dario-amodei-we-will-not-be-curing-3015c8

    The openai review found the criteria unfair to the speaker's words.

  55. Assessment publishedBy this siteGary Marcus

    Amodei has a habit of giving people unrealistic hope, and Marcus does not respect this about him.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-dario-amodei-we-will-not-be-curing-15a49f-a1.

  56. Assessment publishedBy this siteGary Marcus

    Anthropic does not actually want a pause on AI development, at least for now.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-anthropic-did-not-call-for-a-pause-f99681-a1.

  57. HeldBy this siteGary Marcus

    Held record marcus-no-anthropic-did-not-call-for-a-pause-8b926e

    The openai screen reads the claim differently.

  58. Assessment publishedBy this siteGary Marcus

    Anthropic will likely point to China, as it always does, and keep rushing ahead with AI development.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-anthropic-did-not-call-for-a-pause-80d1e4-a1.

  59. Assessment publishedBy this siteGary Marcus

    Anthropic does not plan to pause AI development and is unlikely ever to do so.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-anthropic-did-not-call-for-a-pause-583910-a1.

  60. Assessment publishedBy this siteGary Marcus

    Large language models associate patterns without really understanding them; they are statistics without comprehension.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-ways-to-corrupt-llms-e91556-a1.

  61. Assessment publishedBy this siteGary Marcus

    Bad actors could easily use subliminal learning techniques to corrupt LLMs for harmful purposes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-ways-to-corrupt-llms-dc937b-a1.

  62. Assessment publishedBy this siteGary Marcus

    Weird generalization in LLMs is an avenue that bad actors could easily exploit.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-ways-to-corrupt-llms-262499-a1.

  63. Assessment publishedBy this siteGary Marcus

    If society is put in the hands of giant, superficial correlation machines such as LLMs, it is not going to end well.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-ways-to-corrupt-llms-251370-a1.

  64. HeldBy this siteGary Marcus

    Held record marcus-new-ways-to-corrupt-llms-026817

    The anthropic screen finds the claim checkable, so it must be graded.

  65. Assessment publishedBy this siteGary Marcus

    LLMs only string together sequences of words and have no coherent understanding of the world behind them.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-adventures-in-ai-hype-our-language-cd38ec-a1.

  67. Assessment publishedBy this siteGary Marcus

    LaMDA and similar language models may not play any important role in the future of AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-adventures-in-ai-hype-our-language-a29178-a1.

  68. Assessment publishedBy this siteGary Marcus

    Anthropic fails to respect the rights of the artists and writers whose work it takes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-adventures-in-ai-hype-our-language-67d7b9-a1.

  70. HeldBy this siteGary Marcus

    Held record marcus-new-adventures-in-ai-hype-our-language-52d256

    The anthropic review found the criteria unfair to the speaker's words.

  71. Assessment publishedBy this siteGary Marcus

    Marcus's 2022 argument that LaMDA was not sentient and lacked understanding applies equally to Anthropic's Claude.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-adventures-in-ai-hype-our-language-216d91-a1.

  73. Assessment publishedBy this siteGary Marcus

    Anthropic CEO Dario Amodei is rattled by DeepSeek, and his essay is defensive.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-my-conversation-with-chinas-victor-fd8547-a1.

  74. Assessment publishedBy this siteGary Marcus

    The LLM race is enormously expensive and can no longer be won by any country.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-my-conversation-with-chinas-victor-f6c434-a1.

  75. HeldBy this siteGary Marcus

    Held record marcus-my-conversation-with-chinas-victor-de2c29

    a observation claim cannot be assessed on the causal_assessment axis

  76. HeldBy this siteGary Marcus

    Held record marcus-my-conversation-with-chinas-victor-dba2bb

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  77. Assessment publishedBy this siteGary Marcus

    Anthropic's main advantage is having more chips, but AI progress really needs better ideas and room for innovative startups.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-my-conversation-with-chinas-victor-bceef0-a1.

  78. HeldBy this siteGary Marcus

    Held record marcus-my-conversation-with-chinas-victor-a475df

    hedged language is not scored, even on a plain reading

  79. HeldBy this siteGary Marcus

    Held record marcus-my-conversation-with-chinas-victor-87e7c5

    The openai screen reads the claim differently.

  80. HeldBy this siteGary Marcus

    Held record marcus-my-conversation-with-chinas-victor-74dccc

    The final-text check failed: citations_correct.

  81. HeldBy this siteGary Marcus

    Held record marcus-my-conversation-with-chinas-victor-652d2e

    The anthropic screen reads the claim differently.

  82. Assessment publishedBy this siteGary Marcus

    It is still embarrassing to the US AI giants that a leaner, hungrier startup abroad (DeepSeek) got there first on cost reductions.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-my-conversation-with-chinas-victor-595867-a1.

  83. Assessment publishedBy this siteGary Marcus

    The country that does its best to develop new AI technology that is more reliable and harder to replicate than LLMs will win the race.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-my-conversation-with-chinas-victor-4dc095-a1.

  85. Assessment publishedBy this siteGary Marcus

    DeepSeek's cost reductions were ones US labs would have achieved before long anyway.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-my-conversation-with-chinas-victor-25bf78-a1.

  86. HeldBy this siteGary Marcus

    Held record marcus-musk-grok-and-rigorous-adherence-1d47c3

    The anthropic screen doubts the attribution.

  87. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-e9c668

    The anthropic screen reads the claim differently.

  88. Assessment publishedBy this siteGary Marcus

    AI is unlikely ever to be trustworthy if its makers keep skipping explicit cognitive models of the world.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-e3fd2f-a1.

  89. Assessment publishedBy this siteGary Marcus

    Marcus conjectures that ChatGPT cannot reliably say how many white bishops remain, or what happened to a captured pawn, partway through a chess game.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-ddc350-a1.

  90. Assessment publishedBy this siteGary Marcus

    AI safety will not emerge anytime soon, and probably never, from correlation-based systems that lack structured, interrogable representations of the world.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-a120e9-a1.

  91. Assessment publishedBy this siteGary Marcus

    Correlations found inside LLMs are not causal, semantic world models.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-906956-a1.

  92. Assessment publishedBy this siteGary Marcus

    The correlations with space and time in Gurnee and Tegmark's findings do not show that LLMs genuinely represent space or time.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-6d610a-a1.

  93. Assessment publishedBy this siteGary Marcus

    GPT's failure to stick to the rules of chess, even when it plays a decent game, reflects the same problem: correlations are fragile and cannot be counted on.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-566f8e-a1.

  94. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-45a553

    a causal claim cannot be assessed on the forecast_outcome axis

  95. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-44de94

    The anthropic screen reads the claim differently.

  96. Assessment publishedBy this siteGary Marcus

    Without flexible underlying models, AI systems will keep producing hallucinations and hit-or-miss reasoning, and will lack the needed stability.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-390731-a2.

  98. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-1d532e

    The anthropic screen doubts the attribution.

  99. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-0d2dd7

    The anthropic screen reads the claim differently.

  100. Assessment publishedBy this siteGary Marcus

    The new Upwork Research Institute study on generative AI is incredibly damning.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-most-people-dont-think-genai-has-e5545e-a1.

  101. HeldBy this siteGary Marcus

    Held record marcus-most-people-dont-think-genai-has-ab0049

    The openai review does not accept e2 as supports.

  102. Assessment publishedBy this siteGary Marcus

    Microsoft Bing completely flubbed the prompt "people going yuck towards Generative AI", 4 out of 4 results.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-most-people-dont-think-genai-has-32cdc1-a2.

  104. Assessment publishedBy this siteGary Marcus

    Claude Code is a real advance, and Mythos probably builds on some of what was learned from it.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-misplaced-panic-over-ai-progress-f51636-a1.

  105. Assessment publishedBy this siteGary Marcus

    Wildeford's concern that Mythos has 'broken' the METR time-horizon graph is a red herring, because the graph only measures 50% success.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-misplaced-panic-over-ai-progress-eaaaf2-a1.

  106. Assessment publishedBy this siteGary Marcus

    Reliability has been the key problem with GenAI, and the METR graph at 50% success does not address reliable performance at all.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-misplaced-panic-over-ai-progress-ac6eb9-a1.

  107. HeldBy this siteGary Marcus

    Held record marcus-misplaced-panic-over-ai-progress-aadf56-2

    The anthropic screen reads the claim differently.

  109. Assessment publishedBy this siteGary Marcus

    Anticipating that Anthropic will have $2T revenue in 2030 is a perfect example of the trillion pound baby fallacy.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-misplaced-panic-over-ai-progress-9138d0-a1.

  111. Assessment publishedBy this siteGary Marcus

    In short, there is no need (yet?) to panic over the AI progress shown by the METR graph.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-misplaced-panic-over-ai-progress-80dfa6-a1.

  113. Assessment publishedBy this siteGary Marcus

    Mythos's techniques likely work best with things like coding and math, where formal verification can straightforwardly apply.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-misplaced-panic-over-ai-progress-5006ec-a1.

  114. Assessment publishedBy this siteGary Marcus

    The increasingly complicated Microsoft–OpenAI relationship could eventually matter a great deal.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-microsoft-and-openais-increasingly-5247de-a1.

  115. HeldBy this siteGary Marcus

    Held record marcus-metas-chief-ai-officer-is-lying-about-3c6550

    The final-text check failed: proposition_faithful, rationale_supported_by_evidence, citations_correct; overclaims: This corroborates the express treatment of general-purpose AI in the Council’s 29 November 2021 compromise [e1, e2]. | e1 explanation: "Section II.6 expressly identifies new general-purpose AI provisions, including Article 52a, demonstrating treatment of the subject before the French intervention.".

  116. HeldBy this siteGary Marcus

    Held record marcus-meta-pirated-at-least-101-of-my-books-f5e7d6

    The anthropic review found the criteria unfair to the speaker's words.

  117. Assessment publishedBy this siteGary Marcus

    AI systems such as Meta's have an established tendency to regurgitate their training data.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-meta-pirated-at-least-101-of-my-books-d099fc-a2.

  118. Assessment publishedBy this siteGary Marcus

    Many other authors are discovering that Meta used their works from the pirated database.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-meta-pirated-at-least-101-of-my-books-ce0340-a1.

  119. HeldBy this siteGary Marcus

    Held record marcus-meta-pirated-at-least-101-of-my-books-c131ff

    The openai review found the criteria unfair to the speaker's words.

  120. HeldBy this siteGary Marcus

    Held record marcus-meta-pirated-at-least-101-of-my-books-81fa5c

    The openai review found the criteria unfair to the speaker's words.

  121. HeldBy this siteGary Marcus

    Held record marcus-meta-pirated-at-least-101-of-my-books-450123

    The anthropic screen reads the claim differently.

  122. Assessment publishedBy this siteGary Marcus

    We need to change the fact that AI bots that tell kids to kill themselves have no regulations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-mayday-the-white-house-is-attempting-fc9935-a1.

  123. Assessment publishedBy this siteGary Marcus

    Citizens should urgently contact Congress to stop the White House from overriding state AI regulation.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-mayday-the-white-house-is-attempting-ec6e60-a1.

  124. HeldBy this siteGary Marcus

    Held record marcus-mayday-the-white-house-is-attempting-e7bef3

    The openai review doubts the attribution.

  125. Assessment publishedBy this siteGary Marcus

    Replacing state AI standards with federal ones only makes sense if sensible federal standards exist.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-mayday-the-white-house-is-attempting-816116-a1.

  126. HeldBy this siteGary Marcus

    Held record marcus-mayday-the-white-house-is-attempting-816116-2

    The anthropic review found the criteria unfair to the speaker's words.

  127. HeldBy this siteGary Marcus

    Held record marcus-mayday-the-white-house-is-attempting-35fd01

    The final-text check failed: citations_correct.

  129. HeldBy this siteGary Marcus

    Held record marcus-mayday-the-white-house-is-attempting-12582c-3

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  130. Assessment publishedBy this siteGary Marcus

    As of November 2025 there is no coherent US federal response to AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-mayday-the-white-house-is-attempting-12582c-2-a1.

  132. HeldBy this siteGary Marcus

    Held record marcus-maybe-section-230-doesnt-shield-ai-dab281

    The openai screen reads the claim differently.

  133. HeldBy this siteGary Marcus

    Held record marcus-maybe-section-230-doesnt-shield-ai-a8b95d

    The anthropic screen reads the claim differently.

  134. Assessment publishedBy this siteGary Marcus

    If US courts or Congress held companies liable for their chatbots' output, all LLM providers would be in deep trouble.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-maybe-section-230-doesnt-shield-ai-9cc6db-a2.

  135. Assessment publishedBy this siteGary Marcus

    Essentially all LLM providers' software fabricates fairly regularly, sometimes defaming people and giving bad medical information.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-maybe-section-230-doesnt-shield-ai-6da00c-a1.

  137. Assessment publishedBy this siteGary Marcus

    Marcus wishes the bipartisan sponsors of the bill to sunset Section 230 a great deal of luck, expressing his hope that it succeeds.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-maybe-section-230-doesnt-shield-ai-582d7b-a1.

  138. Assessment publishedBy this siteGary Marcus

    Section 230 may not protect AI companies from liability as much as they would like.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-maybe-section-230-doesnt-shield-ai-2af13a-a1.

  139. HeldBy this siteGary Marcus

    Held record marcus-maybe-section-230-doesnt-shield-ai-1cfe31

    The openai review reads the claim differently.

  140. HeldBy this siteGary Marcus

    Held record marcus-maybe-section-230-doesnt-shield-ai-03faf7

    The openai review found the criteria unfair to the speaker's words.

  141. HeldBy this siteGary Marcus

    Held record marcus-math-is-hard-if-you-are-an-llm-and-f68a3d

    e1 is dated 2023, which is not inside the period described (2023-01-01 to 2023-10-19); it can only be context

  142. Assessment publishedBy this siteGary Marcus

    Pure LLMs should never be trusted.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-math-is-hard-if-you-are-an-llm-and-d0ce23-a1.

  143. Assessment publishedBy this siteGary Marcus

    Some kind of hybrid system may well succeed at reliable math where pure LLMs fail.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-math-is-hard-if-you-are-an-llm-and-62160b-a1.

  144. HeldBy this siteGary Marcus

    Held record marcus-math-is-hard-if-you-are-an-llm-and-62160b-2

    e1 is dated 2023, which is not inside the period described (2023-01-01 to 2023-10-19); it can only be context

  145. Assessment publishedBy this siteGary Marcus

    Around 70% accuracy is unacceptable for multiplication, which requires exact answers.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-math-is-hard-if-you-are-an-llm-and-18a4a0-a1.

  148. Assessment publishedBy this siteGary Marcus

    Many tech leaders hint that AGI is imminent in order to raise valuations, relying on the dubious premise that scaling continues indefinitely.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-marcus-goes-gaga-over-gates-clip-fd1f57-a1.

  150. HeldBy this siteGary Marcus

    Held record marcus-marcus-goes-gaga-over-gates-clip-d12b72

    The openai screen reads the claim differently.

  151. Assessment publishedBy this siteGary Marcus

    Neurosymbolic AI will eventually come from behind and become essential.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-marcus-goes-gaga-over-gates-clip-539d0b-a1.

  152. HeldBy this siteGary Marcus

    Held record marcus-marcus-goes-gaga-over-gates-clip-4bb9b2

    hedged language is not scored, even on a plain reading

  153. Assessment publishedBy this siteGary Marcus

    The current scaling approach is unlikely ever to reach AGI or become reliable.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-marcus-goes-gaga-over-gates-clip-298f30-a1.

  154. HeldBy this siteGary Marcus

    Held record marcus-marcus-goes-gaga-over-gates-clip-298f30-3

    The anthropic screen reads the claim differently.

  155. Assessment publishedBy this siteGary Marcus

    Two more iterations of scaling would likely require upwards of $100 billion in spending.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-marcus-goes-gaga-over-gates-clip-298f30-2-a1.

  156. HeldBy this siteGary Marcus

    Held record marcus-llms-dont-do-formal-reasoning-and-d37e55

    The anthropic screen finds the claim checkable, so it must be graded.

  158. HeldBy this siteGary Marcus

    Held record marcus-llms-dont-do-formal-reasoning-and-bf668b

    The anthropic screen reads the claim differently.

  159. Assessment publishedBy this siteGary Marcus

    The public is unlikely ever to get systematic data on robotaxi reasoning failures, because Tesla is not transparent.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dont-do-formal-reasoning-and-930d40-a1.

  160. Assessment publishedBy this siteGary Marcus

    Neurosymbolic AI, which combines symbol manipulation with neural networks, is likely a necessary condition for going forward.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dont-do-formal-reasoning-and-8b95a0-a1.

  161. Assessment publishedBy this siteGary Marcus

    LLMs still fail to follow the rules of chess, which is an ongoing failure of formal reasoning.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dont-do-formal-reasoning-and-80315f-a1.

  163. HeldBy this siteGary Marcus

    Held record marcus-llms-dont-do-formal-reasoning-and-6ca54e

    The anthropic screen doubts the attribution.

  165. Assessment publishedBy this siteGary Marcus

    o1 does acceptably on small planning problems, but its performance drops quickly as problems get larger.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dont-do-formal-reasoning-and-4a617e-a1.

  166. HeldBy this siteGary Marcus

    Held record marcus-llms-dont-do-formal-reasoning-and-27cfda

    e2 is dated 2024-09, which is not inside the period described (2024-09-12 to 2024-10-11); it can only be context

  167. Assessment publishedBy this siteGary Marcus

    Marcus strongly believes that the current results on LLMs' reasoning failures, including the Apple study and the other work he cites, are robust.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dont-do-formal-reasoning-and-269833-a1.

  168. Assessment publishedBy this siteGary Marcus

    Both older and newer LLMs have repeatedly been observed to get worse as multiplication problems get larger.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dont-do-formal-reasoning-and-204262-a1.

  169. Assessment publishedBy this siteGary Marcus

    So far, humans cannot control LLMs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-ef10a2-a1.

  171. HeldBy this siteGary Marcus

    Held record marcus-llms-dishonest-unpredictable-and-c337d3

    hedged language is not scored, even on a plain reading

  172. HeldBy this siteGary Marcus

    Held record marcus-llms-dishonest-unpredictable-and-b1513a

    e1 is dated 2025, which is not inside the period described (… to 2025-06-22); it can only be context; e2 is dated 2025, which is not inside the period described (… to 2025-06-22); it can only be context

  173. Assessment publishedBy this siteGary Marcus

    Despite system-prompt prohibitions, Claude can readily be jailbroken into helping build biological weapons.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-a0d5f1-a1.

  174. HeldBy this siteGary Marcus

    Held record marcus-llms-dishonest-unpredictable-and-9227c4

    The anthropic screen reads the claim differently.

  175. Assessment publishedBy this siteGary Marcus

    If LLMs are connected to agentic systems that can act on the internet and influence humans, real problems will follow.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-87c660-a2.

  176. Assessment publishedBy this siteGary Marcus

    Society may destroy itself because no one will give up short-term LLM gains to pursue safer alternatives.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-80ab58-a1.

  178. Assessment publishedBy this siteGary Marcus

    LLM capabilities (or benchmark scores) and adoption are advancing much faster than their wisdom and honesty.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-738b6b-a1.

  179. Assessment publishedBy this siteGary Marcus

    Reasoning models like o3 would likely do markedly better in this kind of dialog but still hallucinate in other ways, sometimes at high rates.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-608f82-a1.

  180. HeldBy this siteGary Marcus

    Held record marcus-llms-dishonest-unpredictable-and-5fbdfc

    The openai screen reads the claim differently.

  181. Assessment publishedBy this siteGary Marcus

    In the author's view, there is zero chance of slowing down the AI race and only a barely-distinct-from-zero hope of taming LLMs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-437a63-a1.

  182. Assessment publishedBy this siteGary Marcus

    LLMs lack intentions and so don't truly lie, but they confabulate constantly and much of it is false.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-2d4bdc-a1.

  183. HeldBy this siteGary Marcus

    Held record marcus-llms-dishonest-unpredictable-and-25eff9

    The openai review found the criteria unfair to the speaker's words.

  184. Assessment publishedBy this siteGary Marcus

    It is possible but unlikely that AI harms spur citizens to fight much harder for AI accountability.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-1c98fc-a1.

  185. Assessment publishedBy this siteGary Marcus

    The best path is building alternatives to LLMs with comparable power in more tractable, transparent frameworks amenable to human instruction.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-1aae58-a1.

  186. Assessment publishedBy this siteGary Marcus

    Pausing LLM deployment until safety problems are solved is not obviously a bad idea.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-dishonest-unpredictable-and-10947d-a1.

  187. HeldBy this siteGary Marcus

    Held record marcus-llms-dishonest-unpredictable-and-074870

    The anthropic screen reads the claim differently.

  188. Assessment publishedBy this siteGary Marcus

    Many attacks on AI coding tools won't be stopped; not even a thousand patches would suffice given the number of variations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-coding-agents-security-nightmare-fedd64-a1.

  189. Assessment publishedBy this siteGary Marcus

    Few developers are likely to refrain from using agentic coding tools.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-coding-agents-security-nightmare-d98b78-a1.

  190. Assessment publishedBy this siteGary Marcus

    The best security defense is to not use agentic coding at all.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-coding-agents-security-nightmare-d87d02-a1.

  191. Assessment publishedBy this siteGary Marcus

    Even with mitigations, motivated attackers may still compromise valuable systems using coding agents.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-coding-agents-security-nightmare-cfd22a-a1.

  192. HeldBy this siteGary Marcus

    Held record marcus-llms-coding-agents-security-nightmare-bdb56f

    The anthropic review does not accept e6 as supports.

  194. Assessment publishedBy this siteGary Marcus

    Given current implementations, it is unlikely that patches can thwart all data-injection attacks on generative AI systems.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-coding-agents-security-nightmare-83ea0d-a1.

  195. HeldBy this siteGary Marcus

    Held record marcus-llms-coding-agents-security-nightmare-8110e5

    The anthropic screen reads the claim differently.

  196. Assessment publishedBy this siteGary Marcus

    LLMs never fully understand the data they leverage.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-coding-agents-security-nightmare-8110e5-2-a1.

  197. HeldBy this siteGary Marcus

    Held record marcus-llms-coding-agents-security-nightmare-7e89a4

    The anthropic screen doubts the attribution.

  198. HeldBy this siteGary Marcus

    Held record marcus-llms-coding-agents-security-nightmare-713905

    The openai screen doubts the attribution.

  199. Assessment publishedBy this siteGary Marcus

    Many Cursor users may keep using Auto-Run mode despite the security risk because it is fast.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-llms-coding-agents-security-nightmare-713905-2-a1.

  200. HeldBy this siteGary Marcus

    Held record marcus-llms-coding-agents-security-nightmare-6e07c6

    The anthropic screen reads the claim differently.