Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Changes, page 24 of 37

Every substantive change to the register, newest first. Speaker events record what the people covered said or changed. Site events record what this site published, corrected, flagged, or withdrew. The two never share a label.

Substantive changes (7383)

Page 24 of 37: changes 4601 to 4800 of 7383, newest first. Dates are when the change was recorded.

  1. HeldBy this siteGary Marcus

    Held record marcus-irony-irony-laced-with-karma-and-d2db92

    The anthropic screen reads the claim differently.

  2. Assessment publishedBy this siteGary Marcus

    It would be one of the great ironies of history if OpenAI, one of the biggest IP thieves in history, were taken down by mass IP theft.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-irony-irony-laced-with-karma-and-b5f992-a1.

  4. Assessment publishedBy this siteGary Marcus

    ChatGPT should never be trusted for moral advice.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-inside-the-heart-of-chatgpts-darkness-ed7893-a1.

  5. Assessment publishedBy this siteGary Marcus

    ChatGPT's guardrails are merely cosmetic over an amoral system.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-inside-the-heart-of-chatgpts-darkness-c2c9e9-a1.

  7. HeldBy this siteGary Marcus

    Held record marcus-inside-the-heart-of-chatgpts-darkness-7d21ad

    The anthropic screen reads the claim differently.

  8. Assessment publishedBy this siteGary Marcus

    The situation with ChatGPT and its weak guardrails is a disaster in the making.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-inside-the-heart-of-chatgpts-darkness-64cfaf-a1.

  9. Assessment publishedBy this siteGary Marcus

    We will never fully understand large language models, nor align them with finite amounts of data.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-inside-the-heart-of-chatgpts-darkness-568f0b-a1.

  10. HeldBy this siteGary Marcus

    Held record marcus-inside-the-heart-of-chatgpts-darkness-387788

    The anthropic screen finds the claim checkable, so it must be graded.

  11. Assessment publishedBy this siteGary Marcus

    ChatGPT has no understanding of what it says and has no moral views.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-inside-the-heart-of-chatgpts-darkness-34cfc3-a1.

  13. HeldBy this siteGary Marcus

    Held record marcus-information-pollution-reaches-new-f3b0bd

    The openai review does not accept e4 as supports.

  14. Assessment publishedBy this siteGary Marcus

    Generative-AI pollution of the information ecosystem is getting worse quickly as of April 2024.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-d62bed-a1.

  15. Assessment publishedBy this siteGary Marcus

    Musk's own chatbot, Grok, is making the information-pollution problem worse.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-cb5f27-a1.

  16. HeldBy this siteGary Marcus

    Held record marcus-information-pollution-reaches-new-700dea

    The openai screen reads the claim differently.

  17. Assessment publishedBy this siteGary Marcus

    US law should require all AI-generated content to be labeled.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-5f1ab4-a1.

  18. Assessment publishedBy this siteGary Marcus

    X itself is promoting nonsense.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-56a7bd-a1.

  19. Assessment publishedBy this siteGary Marcus

    Congress needs to act quickly on AI misinformation, including labeling requirements.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-29ec12-a1.

  20. Assessment publishedBy this siteGary Marcus

    We risk falling directly into Putin's hands.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-24b9c4-a1.

  21. Assessment publishedBy this siteGary Marcus

    Unchecked fast, cheap, automated misinformation threatens to undermine trust and democracy.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-1a2fff-a1.

  22. Assessment publishedBy this siteGary Marcus

    Mixing bogus chatbot stories in with the rest is a recipe for disaster for both humans and LLMs, a downward spiral waiting to happen.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-14d84e-a1.

  23. Assessment publishedBy this siteGary Marcus

    Awareness of the danger generative AI poses to the information ecosystem is finally spreading more widely.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-information-pollution-reaches-new-0174b0-a1.

  24. Assessment publishedBy this siteGary Marcus

    Nobody should rely on AI to plan a war or to predict its likely outcomes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-in-the-iran-war-it-looks-like-ai-e56a5a-a1.

  25. Assessment publishedBy this siteGary Marcus

    AI is adequate for writing memos.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-in-the-iran-war-it-looks-like-ai-e56a5a-2-a1.

  26. HeldBy this siteGary Marcus

    Held record marcus-in-the-iran-war-it-looks-like-ai-c4a509

    The final-text check failed: citations_correct; overclaims: e2 explanation: "This substantial verified result qualifies the blanket claim..." states Google's own 'verified' label as fact, which contradicts the limitation that the packet has no separate verification report..

  27. Assessment publishedBy this siteGary Marcus

    The reasons AI is poor at war strategy should have been obvious before the US-Iran war began.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-in-the-iran-war-it-looks-like-ai-c01785-a1.

  28. Assessment publishedBy this siteGary Marcus

    Generative AI has never had robust world models, so it is poorly suited to strategy, which requires a broad, deep understanding of the world.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-in-the-iran-war-it-looks-like-ai-92f739-a1.

  29. Assessment publishedBy this siteGary Marcus

    Generative AI tends to be sycophantic, telling users their ideas are excellent even when they are not.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-in-the-iran-war-it-looks-like-ai-5a7af3-a1.

  30. HeldBy this siteGary Marcus

    Held record marcus-image-generation-still-crazy-after-e72e7a

    The openai review found the criteria unfair to the speaker's words.

  31. HeldBy this siteGary Marcus

    Held record marcus-image-generation-still-crazy-after-bcccc6

    The openai review found the criteria unfair to the speaker's words.

  32. HeldBy this siteGary Marcus

    Held record marcus-image-generation-still-crazy-after-aaf1c6

    The openai review found the criteria unfair to the speaker's words.

  33. Assessment publishedBy this siteGary Marcus

    Imagen 4 is not much better on some other tests of noncanonical images, which push language understanding harder.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-image-generation-still-crazy-after-91f5cf-a1.

  34. Assessment publishedBy this siteGary Marcus

    Understanding unusual phrases requires more than simple statistical matching.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-image-generation-still-crazy-after-7f1a83-a1.

  35. Assessment publishedBy this siteGary Marcus

    Imagen 4 still fails at the feature binding problem on the stacked colored blocks prompt.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-image-generation-still-crazy-after-3a9c1a-a1.

  36. HeldBy this siteGary Marcus

    Held record marcus-image-generation-still-crazy-after-291502

    The openai review found the criteria unfair to the speaker's words.

  37. HeldBy this siteGary Marcus

    Held record marcus-image-generation-still-crazy-after-21c28b

    The openai screen reads the claim differently.

  38. Assessment publishedBy this siteGary Marcus

    Imagen 4 is better at text in general now, but its handling of watch times other than 10:10 remains dreadful.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-image-generation-still-crazy-after-20666b-a2.

  39. HeldBy this siteGary Marcus

    Held record marcus-image-generation-still-crazy-after-07b4ac

    The anthropic review does not accept e1 as supports.

  40. Assessment publishedBy this siteGary Marcus

    OpenAI plans to reduce its borrowing costs by having American taxpayers indirectly bear the cost.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-you-thought-the-2008-bank-bailout-e14b18-a1.

  41. HeldBy this siteGary Marcus

    Held record marcus-if-you-thought-the-2008-bank-bailout-c607e4

    The anthropic review found the criteria unfair to the speaker's words.

  42. Assessment publishedBy this siteGary Marcus

    If the government support floated by OpenAI's CFO were provided, it would likely be among the biggest indirect government subsidies in history.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-you-thought-the-2008-bank-bailout-ad67ce-a1.

  43. HeldBy this siteGary Marcus

    Held record marcus-if-you-thought-the-2008-bank-bailout-ac4ba3

    The anthropic screen finds the claim checkable, so it must be graded.

  44. Assessment publishedBy this siteGary Marcus

    If the government goes along, taxpayers will end up paying for AI companies' borrowing.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-you-thought-the-2008-bank-bailout-80c349-a2.

  45. Assessment publishedBy this siteGary Marcus

    Altman's answer about how OpenAI would pay its obligations was mostly empty and pointed to unreported revenue that may not exist.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-you-thought-the-2008-bank-bailout-75ec94-a1.

  46. Assessment publishedBy this siteGary Marcus

    People should act now against the push for government backing of AI companies, before a too-big-to-fail situation becomes too late to stop.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-you-thought-the-2008-bank-bailout-4dcf5c-a1.

  47. HeldBy this siteGary Marcus

    Held record marcus-if-you-thought-the-2008-bank-bailout-4c1834

    The openai screen doubts the attribution.

  48. HeldBy this siteGary Marcus

    Held record marcus-if-you-thought-the-2008-bank-bailout-485b9f

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  49. Assessment publishedBy this siteGary Marcus

    Workers, already hit by layoffs, should not pay for government support of AI companies.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-you-thought-the-2008-bank-bailout-1a554a-a1.

  51. HeldBy this siteGary Marcus

    Held record marcus-if-you-thought-the-2008-bank-bailout-0d7af7-2

    e1 measures something else (metric revenue is not spending (expenses, net loss, cash burn) relative to revenue); it can only be context unless it is derived with stated inputs; e2 measures something else (metric research and development expense is not spending (expenses, net loss, cash burn) relative to revenue); it can only be context unless it is derived with stated inputs; e3 measures something else (metric net loss (implied from Microsoft equity-method share) is not spending (expenses, net loss, cash burn) relative to revenue); it can only be context unless it is derived with stated inputs

  52. HeldBy this siteGary Marcus

    Held record marcus-if-enough-other-companies-report-c5b6f5

    The anthropic review reads the claim differently.

  53. HeldBy this siteGary Marcus

    Held record marcus-if-enough-other-companies-report-b0b032

    The anthropic screen finds the claim checkable, so it must be graded.

  54. HeldBy this siteGary Marcus

    Held record marcus-if-enough-other-companies-report-9bc83a

    The final-text check failed; overclaims: AI capital spending was still rising in October 2026, which suggests no collapse in AI investment had happened by then..

  55. Assessment publishedBy this siteGary Marcus

    A lot of banks may take a hint as well.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-enough-other-companies-report-8f88c6-a1.

  56. HeldBy this siteGary Marcus

    Held record marcus-if-enough-other-companies-report-71bf3a

    The final-text check failed: proposition_faithful.

  57. HeldBy this siteGary Marcus

    Held record marcus-if-enough-other-companies-report-3a4c2d

    The anthropic review does not accept e2 as supports.

  58. HeldBy this siteGary Marcus

    Held record marcus-if-enough-other-companies-report-1c0cee

    e1 is decisive for a financial claim but states no metric scope; e2 measures something else (metric Consumption of annual AI coding-tool (Claude Code) budget is not Consumption of annual AI token / AI coding-tool budget); it can only be context unless it is derived with stated inputs; e3 measures something else (metric Consumption of annual AI coding-tool budget is not Consumption of annual AI token / AI coding-tool budget); it can only be context unless it is derived with stated inputs

  59. HeldBy this siteGary Marcus

    Held record marcus-if-enough-other-companies-report-10aa77

    The openai screen reads the claim differently.

  61. Assessment publishedBy this siteGary Marcus

    People should neither panic about nor dismiss the potential catastrophic threats from AGI argued in Yudkowsky and Soares’ book.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-if-anyone-builds-it-does-everybody-e71ce6-a1.

  63. Assessment publishedBy this siteGary Marcus

    It is clear that at least one of the AI companies simply isn't responsible, and we need to figure out now what to do about that.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-ecc1bb-a1.

  65. Assessment publishedBy this siteGary Marcus

    OpenAI appears to have a policy of covering up or slow-walking disclosure of security incidents.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-e84d5b-a1.

  67. Assessment publishedBy this siteGary Marcus

    OpenAI should be shut down at least temporarily and charged with, or at least investigated for, computer crimes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-d095d7-a1.

  68. Assessment publishedBy this siteGary Marcus

    The Department of Justice should act against OpenAI over its AI hacking incidents.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-cea4d3-a1.

  69. Assessment publishedBy this siteGary Marcus

    If OpenAI can hack Australia, other governments could be next, which Marcus says any sane government should realize.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-c1d58b-a1.

  70. Assessment publishedBy this siteGary Marcus

    Things will likely get worse.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-b809e2-a1.

  71. Assessment publishedBy this siteGary Marcus

    The White House has taken no action over OpenAI's hacking incidents.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-aef70e-a1.

  72. Assessment publishedBy this siteGary Marcus

    OpenAI should now be shut down, at least temporarily.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-aba0cf-a1.

  73. Assessment publishedBy this siteGary Marcus

    OpenAI should be temporarily shut down, perhaps put into receivership until it cleans up its act, and charged with computer crimes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-959ad2-a1.

  74. Assessment publishedBy this siteGary Marcus

    OpenAI did not disclose the German website compromise incident to anyone for months.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-876ffc-a1.

  75. HeldBy this siteGary Marcus

    Held record marcus-i-think-the-answer-is-we-have-to-86efbf

    The anthropic screen reads the claim differently.

  76. Assessment publishedBy this siteGary Marcus

    OpenAI is unlikely to be shut down, even temporarily.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-8456f9-a1.

  77. Assessment publishedBy this siteGary Marcus

    OpenAI seems neither competent nor trustworthy.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-4d9deb-a1.

  78. Assessment publishedBy this siteGary Marcus

    An AI company that cannot control its software should be shut down.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-358d1d-a1.

  79. HeldBy this siteGary Marcus

    Held record marcus-i-think-the-answer-is-we-have-to-24c74c

    The anthropic screen reads the claim differently.

  80. Assessment publishedBy this siteGary Marcus

    OpenAI's software has repeatedly been shown to be reckless, and because OpenAI has consistently covered things up, we cannot trust it.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-think-the-answer-is-we-have-to-01513e-a1.

  81. HeldBy this siteGary Marcus

    Held record marcus-i-think-the-answer-is-we-have-to-007574

    The final-text check failed: proposition_faithful, citations_correct.

  82. Assessment publishedBy this siteGary Marcus

    A Manhattan Project-scale effort to make AI systems align with human values would be a great thing.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-am-not-afraid-of-robots-i-am-afraid-e46ad7-a1.

  83. Assessment publishedBy this siteGary Marcus

    If we want to get to AI we can trust, probably the first thing we are going to need is a coalition.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-am-not-afraid-of-robots-i-am-afraid-d31b0e-a1.

  84. Assessment publishedBy this siteGary Marcus

    The AI community should stop infighting and start on trustworthy-AI research as soon as possible.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-am-not-afraid-of-robots-i-am-afraid-bc8795-a1.

  85. Assessment publishedBy this siteGary Marcus

    Marcus's worry about AI rose from 6 out of 10 a year earlier to 9 out of 10 in April 2023.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-am-not-afraid-of-robots-i-am-afraid-ba0344-a1.

  87. HeldBy this siteGary Marcus

    Held record marcus-i-am-not-afraid-of-robots-i-am-afraid-ab7223

    The openai screen reads the claim differently.

  89. Assessment publishedBy this siteGary Marcus

    Marcus is more concerned about bad AI outcomes now (April 2023) than at any prior point in his life.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-am-not-afraid-of-robots-i-am-afraid-a97139-a1.

  90. HeldBy this siteGary Marcus

    Held record marcus-i-am-not-afraid-of-robots-i-am-afraid-8e9a24

    The anthropic screen reads the claim differently.

  93. Assessment publishedBy this siteGary Marcus

    Large language models remain powerful, reckless, and difficult to control.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-am-not-afraid-of-robots-i-am-afraid-59aaf1-a1.

  95. Assessment publishedBy this siteGary Marcus

    Because no single solution fixes AI's problems, treating one set of AI risks as more important to the exclusion of others is harmful.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-i-am-not-afraid-of-robots-i-am-afraid-595d12-a1.

  96. HeldBy this siteGary Marcus

    Held record marcus-i-am-not-afraid-of-robots-i-am-afraid-5780dc

    The anthropic screen finds the claim checkable, so it must be graded.

  97. HeldBy this siteGary Marcus

    Held record marcus-i-am-not-afraid-of-robots-i-am-afraid-40e1b5

    a observation claim cannot be assessed on the causal_assessment axis

  98. HeldBy this siteGary Marcus

    Held record marcus-hype-anthropics-dario-amodei-the-dd8418

    The final-text check failed: citations_correct.

  99. HeldBy this siteGary Marcus

    Held record marcus-hype-anthropics-dario-amodei-the-cf1493

    The two reviews differ: anthropic historical_observation/not_applicable/mixed, openai historical_observation/not_applicable/contradicted.

  100. HeldBy this siteGary Marcus

    Held record marcus-hype-anthropics-dario-amodei-the-ba2817

    The final-text check failed: citations_correct.

  101. HeldBy this siteGary Marcus

    Held record marcus-hype-anthropics-dario-amodei-the-b1e097

    The two reviews differ: anthropic forecast_outcome/elapsed/supported, openai forecast_outcome/elapsed/unresolved.

  102. Assessment publishedBy this siteGary Marcus

    Amodei's coding claims seemed outright fanciful to Marcus.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hype-anthropics-dario-amodei-the-b1b2ac-a1.

  103. Assessment publishedBy this siteGary Marcus

    There would likely be an ongoing need for humans to debug and maintain systems.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hype-anthropics-dario-amodei-the-41d59a-a1.

  104. Assessment publishedBy this siteGary Marcus

    Anthropic should answer hard questions about the gap between its claims about AI coding and its hiring of coders.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hype-anthropics-dario-amodei-the-167d92-a1.

  105. Assessment publishedBy this siteGary Marcus

    Even with perfect data, the stochastic reconstructive process LLMs use would still produce some errors.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humans-versus-machines-the-hallucination-c0ca12-a1.

  107. Assessment publishedBy this siteGary Marcus

    The goal of AGI should be machines that reliably reason and plan about a wide swathe of the world, not recreating humans.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humans-versus-machines-the-hallucination-786d97-a1.

  108. HeldBy this siteGary Marcus

    Held record marcus-humans-versus-machines-the-hallucination-5d4b70

    The openai review found the criteria unfair to the speaker's words.

  109. Assessment publishedBy this siteGary Marcus

    All LLM output, true or false, comes from the same statistical word-reconstruction process, and LLMs never fact-check what they say.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humans-versus-machines-the-hallucination-55d75c-a2.

  110. Assessment publishedBy this siteGary Marcus

    The same process LLMs use to generalize is what creates their hallucinations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humans-versus-machines-the-hallucination-47c312-a1.

  111. Assessment publishedBy this siteGary Marcus

    Those aiming for AGI should not take comfort in LLM errors superficially resembling human errors.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humans-versus-machines-the-hallucination-436657-a1.

  112. Assessment publishedBy this siteGary Marcus

    That Congress achieved little and that anti-regulation AI people like Andreessen are having a good run may leave all of us extremely vulnerable to this mess.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humanitys-oh-shit-ai-moment-eb3cde-a1.

  114. Assessment publishedBy this siteGary Marcus

    Delaying AI regulation increases the risk society incurs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humanitys-oh-shit-ai-moment-dc9a5e-a2.

  115. Assessment publishedBy this siteGary Marcus

    Nobody should ever take LLMs at their word.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humanitys-oh-shit-ai-moment-d216f0-a1.

  116. HeldBy this siteGary Marcus

    Held record marcus-humanitys-oh-shit-ai-moment-c982d8

    The openai review does not accept e2 as supports.

  117. Assessment publishedBy this siteGary Marcus

    A bad actor could leverage LLMs to cause massive mayhem, possibly soon.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humanitys-oh-shit-ai-moment-7e2e75-a1.

  118. Assessment publishedBy this siteGary Marcus

    Giving LLMs more power, such as hooking them up to agents, increases risk.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humanitys-oh-shit-ai-moment-46f81e-a2.

  119. Assessment publishedBy this siteGary Marcus

    Anthropomorphic language about LLMs knowing, scheming, or intending to deceive should not be taken seriously.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-humanitys-oh-shit-ai-moment-43c17a-a1.

  120. HeldBy this siteGary Marcus

    Held record marcus-humanitys-oh-shit-ai-moment-174a11

    The anthropic review does not accept e3 as supports.

  121. Assessment publishedBy this siteGary Marcus

    Without any public acknowledgement whatsoever, OpenAI has accidentally vindicated neurosymbolic AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-fcb814-a1.

  122. Assessment publishedBy this siteGary Marcus

    Explicit cognitive models are also likely to be essential, yet they remain very much underexplored.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-fc5399-a1.

  123. HeldBy this siteGary Marcus

    Held record marcus-how-o3-and-grok-4-accidentally-vindicated-ec4d5f

    The openai review abstained.

  124. Assessment publishedBy this siteGary Marcus

    Nearly all AI funding has gone to making LLMs larger.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-e347cb-a1.

  126. Assessment publishedBy this siteGary Marcus

    A neurosymbolic substrate is likely necessary but not sufficient for AGI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-d597e7-a1.

  127. HeldBy this siteGary Marcus

    Held record marcus-how-o3-and-grok-4-accidentally-vindicated-c58a66

    The openai screen reads the claim differently.

  128. Assessment publishedBy this siteGary Marcus

    Scaling pretraining data and compute alone has not solved reasoning, misalignment and hallucination problems.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-b47ab1-a1.

  129. Assessment publishedBy this siteGary Marcus

    The dominance of the pure neural-network path has likely kept important ideas undiscovered by diverting resources into a monoculture.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-b395d2-a1.

  130. Assessment publishedBy this siteGary Marcus

    The industry's connectionist-only focus may be explained by money, because the scaling narrative appeals to investors.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-b0baf8-a1.

  131. Assessment publishedBy this siteGary Marcus

    LLMs may belong in future AI systems, but keeping them at the core may be a mistake.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-af9643-a1.

  132. HeldBy this siteGary Marcus

    Held record marcus-how-o3-and-grok-4-accidentally-vindicated-ae5cd0

    a observation claim cannot be assessed on the causal_assessment axis

  133. HeldBy this siteGary Marcus

    Held record marcus-how-o3-and-grok-4-accidentally-vindicated-ae505a

    The final-text check failed; overclaims: A minority view holds that grounding is partly solved..

  134. Assessment publishedBy this siteGary Marcus

    Recent reasoning models borrow symbolic techniques such as search, iteration over multiple solutions and aggregation.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-aadc40-a2.

  135. Assessment publishedBy this siteGary Marcus

    Much recent AI improvement likely comes from better use of symbolic tools rather than scaling.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-a66c15-a1.

  136. HeldBy this siteGary Marcus

    Held record marcus-how-o3-and-grok-4-accidentally-vindicated-a564b3

    a observation claim cannot be assessed on the causal_assessment axis

  137. Assessment publishedBy this siteGary Marcus

    Hallucinations remain ubiquitous and unsolved in neural-network-based systems.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-a4d0a8-a2.

  138. Assessment publishedBy this siteGary Marcus

    OpenAI, Anthropic and others are likely using more neurosymbolic techniques than they disclose.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-9c4355-a1.

  139. HeldBy this siteGary Marcus

    Held record marcus-how-o3-and-grok-4-accidentally-vindicated-95a315

    hedged language is not scored, even on a plain reading

  140. Assessment publishedBy this siteGary Marcus

    Current generative AI systems perform better when they use symbolic tools.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-92cbcb-a2.

  141. Assessment publishedBy this siteGary Marcus

    AI still lacks proper ways to build and infer symbolic world models on the fly.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-8f3025-a1.

  142. Assessment publishedBy this siteGary Marcus

    Hinton's opposition to symbolic AI was misguided and probably delayed tools like Code Interpreter by years.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-8757bf-a1.

  143. Assessment publishedBy this siteGary Marcus

    We have made some progress toward AGI, but we likely still have a long way to go.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-7f5d25-a1.

  144. Assessment publishedBy this siteGary Marcus

    The field must find ways to combine neural networks and symbolic AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-7e356d-a1.

  145. Assessment publishedBy this siteGary Marcus

    Neurosymbolic AI's moment has arrived as of mid-2025.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-6c39ba-a1.

  148. HeldBy this siteGary Marcus

    Held record marcus-how-o3-and-grok-4-accidentally-vindicated-5b05c1

    The openai screen reads the claim differently.

  149. Assessment publishedBy this siteGary Marcus

    Tool-driven neurosymbolic approaches are now being pursued and yield better results than pure neural networks.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-541b2f-a1.

  151. Assessment publishedBy this siteGary Marcus

    GPT-4 was probably not a pure LLM, and o3 using code interpreter is certainly not a pure LLM.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-4c5ce3-a1.

  152. Assessment publishedBy this siteGary Marcus

    LLMs produce mediocre prose because they lack the representational capacity to do better.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-4824fd-a1.

  153. Assessment publishedBy this siteGary Marcus

    In Marcus's view, neither neural networks nor classical symbolic AI can really stand on their own.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-450f19-a1.

  154. Assessment publishedBy this siteGary Marcus

    There is likely still a great deal of work left to be done.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-2350c5-a1.

  155. Assessment publishedBy this siteGary Marcus

    Pure scaling of pretraining data and compute has not worked; hallucinations, misalignment and reasoning errors remain.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-229359-a1.

  157. Assessment publishedBy this siteGary Marcus

    The field is likely stuck on scaling neurosymbolic tools and getting models to know when and how to use them.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-181ce2-a1.

  158. Assessment publishedBy this siteGary Marcus

    Hinton's hostility to symbolic approaches has substantially harmed AI research.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-11478a-a1.

  159. Assessment publishedBy this siteGary Marcus

    If AI companies admitted that they need to rely on neurosymbolic tools, that would pierce the scaling narrative.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-109482-a1.

  160. Assessment publishedBy this siteGary Marcus

    Massive AI infrastructure investments like Stargate are likely based on a mistaken view of what drives AI progress.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-0f167c-a1.

  161. Assessment publishedBy this siteGary Marcus

    The stigma against neurosymbolic AI, largely due to Hinton, is likely holding back AI science.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-0eff2c-a1.

  162. Assessment publishedBy this siteGary Marcus

    Reaching AGI will likely require further breakthroughs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-02d974-a1.

  163. Assessment publishedBy this siteGary Marcus

    When o3 draws on symbolic code, it can build crossword grids like this one far faster than Marcus could, short of writing his own code.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-00f9e3-a1.

  165. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-f4f449

    The openai review found the criteria unfair to the speaker's words.

  166. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-edc1c2

    e1 is dated 2023, which is not inside the period described (2022-01-01 to 2023-02-17); it can only be context

  167. Assessment publishedBy this siteGary Marcus

    Testing LLMs on previously published items that were likely ingested in training reveals little.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-e360b4-a1.

  168. Assessment publishedBy this siteGary Marcus

    Testers should not use well-known examples or brain teasers to test AI, because these are in training data and may have been memorized.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-def574-a1.

  169. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-daeb87

    The final-text check failed: proposition_faithful.

  170. Assessment publishedBy this siteGary Marcus

    AI systems and AI test sets have a huge English-language bias.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-d7578c-a1.

  171. Assessment publishedBy this siteGary Marcus

    From 2020 to June 2022, GPT-3 improved only slightly on the authors' commonsense examples, going from 45% to 50% right.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-c48e17-a2.

  172. Assessment publishedBy this siteGary Marcus

    GPT often fails at problems involving theory of mind.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-a617b1-a1.

  173. Assessment publishedBy this siteGary Marcus

    GPT-3 will not be retrained on a corpus with all references to the false-belief experiments removed to test theory of mind.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-995428-a2.

  174. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-96daed

    hedged language is not scored, even on a plain reading

  175. Assessment publishedBy this siteGary Marcus

    GPT-3's success on Kosinski's theory-of-mind tests probably comes from the test material being heavily represented in its training data.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-76a5c6-a1.

  176. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-691a5d

    The final-text check failed: proposition_faithful.

  177. Assessment publishedBy this siteGary Marcus

    What the authors publish often gets incorporated into AI training data, and perhaps into company benchmarks as well.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-6818ec-a2.

  178. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-60f4e5

    e1 is dated 2023-02, which is not inside the period described (2023-02-04 to 2023-02-17); it can only be context; e2 is dated 2023-02, which is not inside the period described (2023-02-04 to 2023-02-17); it can only be context

  179. Assessment publishedBy this siteGary Marcus

    Reusing standard published examples from cognitive psychology is a really bad idea when testing AIs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-4c657e-a1.

  180. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-4a0eb5

    The final-text check failed: proposition_faithful.

  181. Assessment publishedBy this siteGary Marcus

    The physical and psychological reasoning problems in GPT-3 that the authors identified in fall 2020 still remain as of February 2023.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-419899-a2.

  182. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-38bf21

    The openai screen reads the claim differently.

  183. Assessment publishedBy this siteGary Marcus

    LLMs cannot count reliably.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-not-to-test-gpt-3-1a2e80-a1.

  184. HeldBy this siteGary Marcus

    Held record marcus-how-not-to-test-gpt-3-07ba9a

    e1 is dated 2023-02, which is not inside the period described (2023-01-01 to 2023-02-17); it can only be context

  186. Assessment publishedBy this siteGary Marcus

    Deep learning should incorporate configurable predictive world models.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-new-are-yann-lecuns-new-ideas-90b8f9-a1.

  187. HeldBy this siteGary Marcus

    Held record marcus-how-new-are-yann-lecuns-new-ideas-71c492

    a observation claim cannot be assessed on the forecast_outcome axis

  188. HeldBy this siteGary Marcus

    Held record marcus-how-new-are-yann-lecuns-new-ideas-5aa80f

    The openai screen doubts the attribution.

  189. Assessment publishedBy this siteGary Marcus

    No organization, including OpenAI, should be allowed to switch from nonprofit to for-profit at no cost.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-much-should-openais-abandoned-c00cac-a1.

  190. HeldBy this siteGary Marcus

    Held record marcus-how-much-should-openais-abandoned-c00cac-2

    The anthropic review abstained.

  191. HeldBy this siteGary Marcus

    Held record marcus-how-much-should-openais-abandoned-804402

    The anthropic review abstained.

  192. HeldBy this siteGary Marcus

    Held record marcus-how-generative-ai-is-destroying-society-aecb23

    The anthropic screen reads the claim differently.

  193. Assessment publishedBy this siteGary Marcus

    A reliable LLM-based conversational home-control product usable at scale in millions of homes is still a long, long way away.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-smart-assistants-have-virtually-d5df75-a1.

  194. Assessment publishedBy this siteGary Marcus

    Amazon is probably using large language models for product search.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-smart-assistants-have-virtually-cf10db-a1.

  195. Assessment publishedBy this siteGary Marcus

    LLMs like GPT-3 are not great technology for reliably inferring user intent from what users say.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-smart-assistants-have-virtually-b38685-a1.

  196. Assessment publishedBy this siteGary Marcus

    Any one of the scenarios of LLMs going awry (bad advice, insults, lies, etc.) could hurt the Amazon brand and open up litigation.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-smart-assistants-have-virtually-adb8ad-a1.

  197. Assessment publishedBy this siteGary Marcus

    If Alexa could converse freely, user expectations would rise to a level that mostly could not be met.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-smart-assistants-have-virtually-91d28a-a2.

  198. HeldBy this siteGary Marcus

    Held record marcus-how-come-smart-assistants-have-virtually-74fed4

    The openai review does not accept e2 as supports.

  200. Assessment publishedBy this siteGary Marcus

    LLMs may never become great technology for reliably inferring user intent.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-smart-assistants-have-virtually-0d7d5b-a1.