Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Changes, page 25 of 37

Every substantive change to the register, newest first. Speaker events record what the people covered said or changed. Site events record what this site published, corrected, flagged, or withdrew. The two never share a label.

Substantive changes (7383)

Page 25 of 37: changes 4801 to 5000 of 7383, newest first. Dates are when the change was recorded.

  1. Assessment publishedBy this siteGary Marcus

    Marcus believes his essay 'Deep Learning is Hitting a Wall' is still incredibly relevant, even with all the advances of the last 8 months.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-gpt-can-seem-so-brilliant-fc6d04-a1.

  2. Assessment publishedBy this siteGary Marcus

    GPT-3 is a mimic that does not understand what it is saying.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-gpt-can-seem-so-brilliant-f0db46-a1.

  3. HeldBy this siteGary Marcus

    Held record marcus-how-come-gpt-can-seem-so-brilliant-dff37d

    The anthropic review does not accept e1 as supports.

  4. HeldBy this siteGary Marcus

    Held record marcus-how-come-gpt-can-seem-so-brilliant-ba6d55

    The anthropic screen reads the claim differently.

  5. Assessment publishedBy this siteGary Marcus

    GPT sounds plausible because each piece it combines comes from something humans actually said, and the pieces are often loosely related.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-gpt-can-seem-so-brilliant-57cc2b-a1.

  6. Assessment publishedBy this siteGary Marcus

    GPT's output is not random, because it recombines things real people have said.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-gpt-can-seem-so-brilliant-33bab9-a1.

  7. HeldBy this siteGary Marcus

    Held record marcus-how-come-gpt-can-seem-so-brilliant-1a85ab

    The anthropic review does not accept e1 as supports.

  8. Assessment publishedBy this siteGary Marcus

    GPT-3 does not understand how the world works.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-come-gpt-can-seem-so-brilliant-1866b5-a1.

  9. Assessment publishedBy this siteGary Marcus

    There is currently no legal requirement in the West for AI companies to resolve truthfulness or alignment issues.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-china-might-crush-the-west-in-f419e7-a1.

  10. HeldBy this siteGary Marcus

    Held record marcus-how-china-might-crush-the-west-in-e8ae9f

    The openai screen reads the claim differently.

  11. HeldBy this siteGary Marcus

    Held record marcus-how-china-might-crush-the-west-in-e2417a

    The anthropic screen finds the claim checkable, so it must be graded.

  12. HeldBy this siteGary Marcus

    Held record marcus-how-china-might-crush-the-west-in-df4765

    e2 is dated 2023-04, which is not inside the period described (2023-01-01 to 2023-04-25); it can only be context

  13. Assessment publishedBy this siteGary Marcus

    If needed, AI should be regulated rather than left to corporations with only nominal commitment to responsible AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-china-might-crush-the-west-in-ddffa6-a1.

  15. Assessment publishedBy this siteGary Marcus

    Neurosymbolic AI is out of fashion, not well developed, and not strongly supported by venture capital.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-china-might-crush-the-west-in-d81788-2-a1.

  17. HeldBy this siteGary Marcus

    Held record marcus-how-china-might-crush-the-west-in-8ebd5b

    The openai screen reads the claim differently.

  18. Assessment publishedBy this siteGary Marcus

    The West should counter China partly by choosing innovation-encouraging AI regulations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-china-might-crush-the-west-in-8a4b8d-a1.

  19. HeldBy this siteGary Marcus

    Held record marcus-how-china-might-crush-the-west-in-595ea5

    The final-text check failed: citations_correct.

  20. HeldBy this siteGary Marcus

    Held record marcus-how-china-might-crush-the-west-in-53b72f

    The openai screen reads the claim differently.

  21. Assessment publishedBy this siteGary Marcus

    Section 230 should not shield platforms from liability for misinformation generated by their own AI tools.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-china-might-crush-the-west-in-4beea8-a1.

  22. Assessment publishedBy this siteGary Marcus

    Regulations that spur greater accuracy might actually spur greater innovation.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-china-might-crush-the-west-in-34aa9f-a1.

  23. Assessment publishedBy this siteGary Marcus

    When true AGI arrives, warnings may be ignored due to earlier false alarms.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-agi-is-nigh-doomers-own-goaled-ef5f46-a1.

  25. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-d869b2

    The anthropic screen reads the claim differently.

  27. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-c9ea80

    The anthropic screen doubts the attribution.

  28. Assessment publishedBy this siteGary Marcus

    Government use of unreliable AI in war may accidentally accelerate war.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-agi-is-nigh-doomers-own-goaled-c0ecdf-a1.

  29. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-aa5fa5

    The anthropic screen finds the claim checkable, so it must be graded.

  30. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-a674ed

    The final-text check failed; overclaims: In a tournament of eight leading LLMs, most first-round matches had more games lost to illegal moves than to checkmate..

  32. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-8da174

    The anthropic screen reads the claim differently.

  33. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-7e1f6e

    The anthropic screen reads the claim differently.

  34. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-67ebe8

    The anthropic review reads the claim differently.

  35. Assessment publishedBy this siteGary Marcus

    Preparing for the AI future should begin with a realistic assessment of current AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-agi-is-nigh-doomers-own-goaled-604443-a1.

  36. Assessment publishedBy this siteGary Marcus

    History will probably regard the Pentagon's rejection of Anthropic and contract with OpenAI as a turning point.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-agi-is-nigh-doomers-own-goaled-4b58ac-a1.

  37. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-4085ea

    The openai screen reads the claim differently.

  38. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-23d009

    The final-text check failed; overclaims: Company claim that an LLM assistant handled customer chats on its own at scale..

  40. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-165b39

    The openai review reads the claim differently.

  41. HeldBy this siteGary Marcus

    Held record marcus-how-agi-is-nigh-doomers-own-goaled-0fea27

    The anthropic screen finds the claim checkable, so it must be graded.

  42. Assessment publishedBy this siteGary Marcus

    Doomer AGI projections have been influential, and not in a good way.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-how-agi-is-nigh-doomers-own-goaled-0956cd-a1.

  43. Assessment publishedBy this siteGary Marcus

    The continued absence of a GPT-5 level model from OpenAI or competitors is evidence that LLM development may have reached a phase of diminishing returns.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-openais-new-gpt-4o-df2268-a1.

  44. Assessment publishedBy this siteGary Marcus

    OpenAI does not have GPT-5 after 14 months of trying since GPT-4.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-openais-new-gpt-4o-d073ef-a1.

  46. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-openais-new-gpt-4o-b9d008

    The final-text check failed: citations_correct; overclaims: its evaluation prose also describes parity with Turbo in text, reasoning, and coding..

  47. Assessment publishedBy this siteGary Marcus

    Lots of quirky errors in GPT-4o are already being reported, same as ever, such as a reasoning error reported by Jane Rosenzweig.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-openais-new-gpt-4o-53a025-a1.

  48. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-openais-new-gpt-4o-48cb2a

    The anthropic screen finds the claim checkable, so it must be graded.

  49. Assessment publishedBy this siteGary Marcus

    If OpenAI had GPT-5, it would have shown it.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-openais-new-gpt-4o-2b9087-a1.

  50. Assessment publishedBy this siteGary Marcus

    Commercially, GPT-4 is no longer a unique offering.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-googles-gemini-and-gpt-be8f17-a3.

  51. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-googles-gemini-and-gpt-9ab98a

    The final-text check failed: citations_correct.

  52. Assessment publishedBy this siteGary Marcus

    Gates and Altman have both been dropping hints, and GPT-5 has not arrived after a year despite immense commercial desire.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-googles-gemini-and-gpt-96c6b0-a2.

  53. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-googles-gemini-and-gpt-54d938

    The openai review found the criteria unfair to the speaker's words.

  54. Assessment publishedBy this siteGary Marcus

    The fact that Google, with all its resources, did not blow away GPT-4 could be telling about whether LLMs are close to a plateau.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-googles-gemini-and-gpt-370cb6-a1.

  55. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-googles-gemini-3-d0a7cf

    The final-text check failed: citations_correct.

  56. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-googles-gemini-3-c040fa

    The anthropic review does not accept e3 as contradicts.

  57. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-googles-gemini-3-724a01

    The openai review found the criteria unfair to the speaker's words.

  58. Assessment publishedBy this siteGary Marcus

    Gemini 3 is better than any other model on most measures.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-googles-gemini-3-53bbf6-a1.

  59. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-googles-gemini-3-10d740

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  60. Assessment publishedBy this siteGary Marcus

    The idea that any AI infrastructure profits will much help the American people, rather than just enrich the owners, is entirely speculative.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-an-ai-catfight-efd2ec-a1.

  61. Assessment publishedBy this siteGary Marcus

    Musk could be right that Altman is overstating the Stargate funding, the same way OpenAI's original $1B funding claim was overstated.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-an-ai-catfight-ef42ce-a1.

  62. Assessment publishedBy this siteGary Marcus

    The idea that LLMs or other OpenAI technology will be enormously profitable is entirely speculative.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-an-ai-catfight-9bb667-a1.

  63. Assessment publishedBy this siteGary Marcus

    The idea that building the infrastructure will give the U.S. a long-term AI advantage over China is speculative, since China is trying to build the same.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-an-ai-catfight-97db07-a1.

  64. Assessment publishedBy this siteGary Marcus

    Altman's claim that Stargate will be great for the country rests on several conjectures, as much of what he says does.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-an-ai-catfight-09bae6-a1.

  65. Assessment publishedBy this siteGary Marcus

    Read between the lines, the White House Accord means the signing AI companies agree not to be regulated.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-on-a-weak-white-house-accord-bd6a35-a1.

  66. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-a-weak-white-house-accord-8ee89c

    The openai screen doubts the attribution.

  67. HeldBy this siteGary Marcus

    Held record marcus-hot-take-on-a-weak-white-house-accord-6f7114

    The anthropic screen finds the claim checkable, so it must be graded.

  68. HeldBy this siteGary Marcus

    Held record marcus-hot-take-gpt-45-is-a-nothing-burger-fe32a7

    The anthropic screen finds the claim checkable, so it must be graded.

  69. Assessment publishedBy this siteGary Marcus

    Pure scaling of LLMs has hit a wall or at least massively diminishing returns.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-gpt-45-is-a-nothing-burger-fa4620-a2.

  70. Assessment publishedBy this siteGary Marcus

    Marcus's 2024 predictions still hold, supported by Grok 3's underwhelming performance despite its massive scale.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-gpt-45-is-a-nothing-burger-f79eb6-a1.

  71. Assessment publishedBy this siteGary Marcus

    Neither Grok 3 nor GPT-4.5 (probably originally intended as GPT-5) fundamentally changed the state of AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-gpt-45-is-a-nothing-burger-ec113f-a1.

  72. HeldBy this siteGary Marcus

    Held record marcus-hot-take-gpt-45-is-a-nothing-burger-ea651f

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  73. Assessment publishedBy this siteGary Marcus

    GPT-4.5 did not eliminate hallucinations or stupid errors.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-gpt-45-is-a-nothing-burger-e8dbb7-a1.

  74. HeldBy this siteGary Marcus

    Held record marcus-hot-take-gpt-45-is-a-nothing-burger-c98abc

    The anthropic screen finds the claim checkable, so it must be graded.

  75. Assessment publishedBy this siteGary Marcus

    Scaling data and compute is not a physical law, and pretty much everything Marcus has told his readers was true.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-gpt-45-is-a-nothing-burger-adef4c-a1.

  76. Assessment publishedBy this siteGary Marcus

    The hype about GPT-5 over the past couple of years has not proven true.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-gpt-45-is-a-nothing-burger-8dc9e3-a1.

  77. Assessment publishedBy this siteGary Marcus

    Sam Altman was unusually tepid in announcing GPT-4.5, not mentioning AGI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hot-take-gpt-45-is-a-nothing-burger-71018b-a2.

  78. HeldBy this siteGary Marcus

    Held record marcus-hot-take-gpt-45-is-a-nothing-burger-47659e

    The anthropic screen reads the claim differently.

  79. HeldBy this siteGary Marcus

    Held record marcus-hot-take-gpt-45-is-a-nothing-burger-41ce40

    The anthropic screen finds the claim checkable, so it must be graded.

  80. HeldBy this siteGary Marcus

    Held record marcus-hot-take-gpt-45-is-a-nothing-burger-07827b

    The openai screen reads the claim differently.

  81. Assessment publishedBy this siteGary Marcus

    Generative AI has made no progress on factuality and compositionality since 2022, despite progress in other respects.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-redux-f5ef21-a1.

  82. Assessment publishedBy this siteGary Marcus

    DALL-E 2 is not as smart as it appears.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-redux-c336ee-a1.

  83. Assessment publishedBy this siteGary Marcus

    Gemini can draw a horse riding an astronaut, but getting it to do so is very difficult.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-redux-c1d0b0-a1.

  84. HeldBy this siteGary Marcus

    Held record marcus-horse-rides-astronaut-redux-a405cb

    The anthropic screen reads the claim differently.

  85. Assessment publishedBy this siteGary Marcus

    DALL-E's failure on 'horse rides astronaut' was due to deficient language understanding, not illustration ability.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-redux-a310b4-a1.

  86. Assessment publishedBy this siteGary Marcus

    After several retries, Gemini produced only one correct horse-riding-astronaut image out of three.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-redux-43e121-a1.

  87. Assessment publishedBy this siteGary Marcus

    The machine learning community has recently made premature declarations and may not have learned the Clever Hans lesson about overclaiming.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-d5bd09-a1.

  88. Assessment publishedBy this siteGary Marcus

    DALL-E 2 is not as smart as it appears.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-c336ee-a1.

  89. Assessment publishedBy this siteGary Marcus

    Imagen has no understanding of weight, mass or density.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-a6cd19-a1.

  90. HeldBy this siteGary Marcus

    Held record marcus-horse-rides-astronaut-889e7b

    The final-text check failed: proposition_faithful.

  91. Assessment publishedBy this siteGary Marcus

    Imagen almost certainly does not have a deep understanding of language.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-792402-a1.

  92. Assessment publishedBy this siteGary Marcus

    Imagen's failure on 'a horse riding an astronaut' was not due to common sense.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-6501e4-a2.

  94. Assessment publishedBy this siteGary Marcus

    DALL-E 2 is easily fooled, especially by complex sentences and words requiring deep language understanding.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-61347d-a1.

  95. Assessment publishedBy this siteGary Marcus

    Imagen lacks deep understanding of language despite Google's PR.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-horse-rides-astronaut-4f1bbd-a1.

  96. HeldBy this siteGary Marcus

    Held record marcus-horse-rides-astronaut-249725

    a causal claim cannot be assessed on the forecast_outcome axis

  98. HeldBy this siteGary Marcus

    Held record marcus-horse-rides-astronaut-20af65

    e2 is dated 2022-05, which is not inside the period described (2022-05-01 to 2022-05-28); it can only be context

  101. Assessment publishedBy this siteGary Marcus

    Humanity will regret it if it does not heed Bengio's warnings about AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-historic-un-security-council-briefing-d49f26-a1.

  102. HeldBy this siteGary Marcus

    Held record marcus-historic-un-security-council-briefing-a3f21d

    e4 is dated 2026-09-24, which is not inside the period described (2026-09-23 to 2026-09-23); it can only be context

  103. Assessment publishedBy this siteGary Marcus

    AI systems should undergo safety testing before deployment.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-historic-un-security-council-briefing-8f7a38-a1.

  104. Assessment publishedBy this siteGary Marcus

    There is a need for liability in AI governance.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-historic-un-security-council-briefing-7c6294-a1.

  105. HeldBy this siteGary Marcus

    Held record marcus-historic-un-security-council-briefing-7b262f

    The anthropic screen reads the claim differently.

  106. Assessment publishedBy this siteGary Marcus

    Action on AI governance is needed immediately.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-historic-un-security-council-briefing-37cbe6-a1.

  107. HeldBy this siteGary Marcus

    Held record marcus-historic-un-security-council-briefing-2ddee6

    The openai review found the criteria unfair to the speaker's words.

  108. Assessment publishedBy this siteGary Marcus

    AI governance requires global dialogue and international cooperation.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-historic-un-security-council-briefing-2a55c8-a1.

  109. Assessment publishedBy this siteGary Marcus

    AI systems should be subject to transparency auditing.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-historic-un-security-council-briefing-07350b-a1.

  111. HeldBy this siteGary Marcus

    Held record marcus-hinton-vs-lecun-vs-ng-vs-tegmark-f67dde

    The openai screen reads the claim differently.

  112. Assessment publishedBy this siteGary Marcus

    The EU's decision on regulating generative AI will affect people everywhere, not just in the EU.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hinton-vs-lecun-vs-ng-vs-tegmark-265d63-a1.

  113. Assessment publishedBy this siteGary Marcus

    People who know anyone in the EU government should tell them now that they do not want big tech to run the world.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hinton-vs-lecun-vs-ng-vs-tegmark-0e26da-a1.

  114. Assessment publishedBy this siteGary Marcus

    Nearly everything Tegmark said that week about AI regulation was true.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hinton-vs-lecun-vs-ng-vs-tegmark-0373b4-a1.

  115. HeldBy this siteGary Marcus

    Held record marcus-hinton-vs-lecun-vs-ng-vs-tegmark-030332

    The openai screen reads the claim differently.

  116. Assessment publishedBy this siteGary Marcus

    Multimodal models, like text-only ones before them, depend on correlations (now between words and pixels) instead of understanding.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-eed193-a1.

  118. Assessment publishedBy this siteGary Marcus

    Multimodal training is not making hallucinations go away.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-d71f3a-a2.

  119. HeldBy this siteGary Marcus

    Held record marcus-hello-multimodal-hallucinations-ce95ba

    The anthropic screen finds the claim checkable, so it must be graded.

  120. Assessment publishedBy this siteGary Marcus

    When questioned about the kitchen image, ChatGPT's vision model left out drawers that are there and made up a kitchen island that is not there.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-ce5317-a1.

  121. Assessment publishedBy this siteGary Marcus

    LLMs lack stable world models, which makes them poor at planning and unable to reason reliably.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-c3ae37-a1.

  122. Assessment publishedBy this siteGary Marcus

    GPT-4V can use words like refrigerator and counter, but that does not show it knows what they mean.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-8139b8-a1.

  123. Assessment publishedBy this siteGary Marcus

    Hallucinations in large language models will remain inevitable.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-773cbd-a1.

  124. Assessment publishedBy this siteGary Marcus

    As is by now widely known, hallucinations are a hallmark of systems that generalize based on word distribution.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-5f7eea-a1.

  125. Assessment publishedBy this siteGary Marcus

    As far as the authors know, no LLM-based system can yet reliably do the blocks-world modeling that SHRDLU attempted in the 1970s.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-1c3721-a1.

  126. Assessment publishedBy this siteGary Marcus

    Even if this specific kitchen example gets patched, many other similar pictures will probably still produce similar errors in multimodal LLMs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hello-multimodal-hallucinations-07678a-a1.

  127. HeldBy this siteGary Marcus

    Held record marcus-have-llms-improved-patient-outcomes-c1f91e

    The two reviews differ: anthropic forecast_outcome/open_horizon/unresolved, openai forecast_outcome/conclusive_event/supported.

  128. Assessment publishedBy this siteGary Marcus

    Current AI tools such as domain-general chatbots may not be adequate for the job of improving medicine.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-have-llms-improved-patient-outcomes-c1f91e-2-a1.

  129. Assessment publishedBy this siteGary Marcus

    The motivations for allowing H200 sales to China are, at best, unclear.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-f5f522-a1.

  130. Assessment publishedBy this siteGary Marcus

    The US's large investments in AI infrastructure may not pay off.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-e453a8-a1.

  131. Assessment publishedBy this siteGary Marcus

    Proposed federal preemption of states' rights to regulate AI is unpopular with both US political parties.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-e34087-a1.

  132. Assessment publishedBy this siteGary Marcus

    Writing with ChatGPT is outsourcing education, in the context of college education.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-d20f92-a1.

  134. Assessment publishedBy this siteGary Marcus

    The decision to allow H200 sales to China is unpopular with both US political parties.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-a5d127-a1.

  135. Assessment publishedBy this siteGary Marcus

    China's reluctance to stock up on H200s may also be a sign that China has realized GPUs aren't the royal road to AGI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-8a4453-a1.

  136. HeldBy this siteGary Marcus

    Held record marcus-has-the-chinese-government-figured-6c6c23

    The openai screen reads the claim differently.

  137. HeldBy this siteGary Marcus

    Held record marcus-has-the-chinese-government-figured-665d8f

    The openai review found the criteria unfair to the speaker's words.

  138. Assessment publishedBy this siteGary Marcus

    China's reluctance to buy H200s is partly driven by protectionism favoring homegrown companies like Huawei.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-47880d-a1.

  139. Assessment publishedBy this siteGary Marcus

    Trump's decision to allow Nvidia to sell H200 chips to China has not been well received.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-the-chinese-government-figured-20d843-a1.

  140. HeldBy this siteGary Marcus

    Held record marcus-has-the-bailout-of-generative-ai-e9002b

    The anthropic screen doubts the attribution.

  142. HeldBy this siteGary Marcus

    Held record marcus-has-the-bailout-of-generative-ai-39d826

    The anthropic screen doubts the attribution.

  144. Assessment publishedBy this siteGary Marcus

    AI research should stop pursuing the current LLM/scaling approach as soon as possible and look for new paradigms.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-sam-altman-gone-full-gary-marcus-b15a37-a1.

  145. Assessment publishedBy this siteGary Marcus

    As of November 2023, the hubris of spring 2022 has diminished to some small but detectable extent.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-sam-altman-gone-full-gary-marcus-b0923d-a1.

  146. HeldBy this siteGary Marcus

    Held record marcus-has-sam-altman-gone-full-gary-marcus-63afdb

    The anthropic review does not accept e4 as contradicts.

  147. HeldBy this siteGary Marcus

    Held record marcus-has-sam-altman-gone-full-gary-marcus-208fb5

    The openai review found the criteria unfair to the speaker's words.

  148. HeldBy this siteGary Marcus

    Held record marcus-has-sam-altman-gone-full-gary-marcus-18d1cc

    The openai screen reads the claim differently.

  149. Assessment publishedBy this siteGary Marcus

    If Elon Musk could not bend Grok to his will back in February 2025, he has presumably kept trying to do so since then.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-grok-lost-its-mind-and-mind-melded-c41c77-a1.

  150. Assessment publishedBy this siteGary Marcus

    People should be worried, emphatically so, about powerful people using LLMs to shape their ideas.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-grok-lost-its-mind-and-mind-melded-27a1b1-a1.

  151. HeldBy this siteGary Marcus

    Held record marcus-has-google-gone-too-woke-why-even-e97e57

    The anthropic screen reads the claim differently.

  152. Assessment publishedBy this siteGary Marcus

    Despite billions more in investment and bigger models, the overactive guardrail problem in AI chatbots has not been solved as of February 2024.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-e9596a-a2.

  154. Assessment publishedBy this siteGary Marcus

    Gemini's historically inaccurate diversity in generated images is a general issue, not limited to popes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-83060f-a1.

  155. Assessment publishedBy this siteGary Marcus

    Making AI culturally sensitive yet historically accurate is hard because of skewed data/cultural change and AI's lack of intelligence.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-7cf71d-a1.

  156. Assessment publishedBy this siteGary Marcus

    Current AI is not capable of getting the balance between cultural sensitivity and historical accuracy right.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-7079cc-a1.

  157. Assessment publishedBy this siteGary Marcus

    Current AI systems mix historical and current data indiscriminately and are too unsophisticated to determine a representative current sample.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-6d5f86-a1.

  158. Assessment publishedBy this siteGary Marcus

    Increased transparency about LLM training data and hidden system prompts is essential.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-693c73-a1.

  159. Assessment publishedBy this siteGary Marcus

    Current AI systems clearly lack a nuanced understanding of what is being asked.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-662d07-a1.

  160. Assessment publishedBy this siteGary Marcus

    It is good that Google is trying to counter minority underrepresentation in generated images, though it is doing so ineptly.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-5e9091-a1.

  161. Assessment publishedBy this siteGary Marcus

    Gemini's overly restrictive responses to some queries will likely be partly patched soon.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-has-google-gone-too-woke-why-even-510ab1-a1.

  162. HeldBy this siteGary Marcus

    Held record marcus-has-google-gone-too-woke-why-even-510ab1-2

    The openai review found the criteria unfair to the speaker's words.

  165. HeldBy this siteGary Marcus

    Held record marcus-has-google-gone-too-woke-why-even-262cdc

    e1 is dated 2024-02, which is not inside the period described (2022-12-25 to 2024-02-21); it can only be context

  166. Assessment publishedBy this siteGary Marcus

    AI needs hybrid systems that include LLMs as one tool among many.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-e5fb10-a1.

  167. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-e5fb10-2

    The anthropic screen reads the claim differently.

  168. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-dc7614

    The final-text check failed: conditions_shown, rationale_supported_by_evidence, citations_correct; overclaims: OpenAI also says the campaigns gained little reach, but the claim is only that the use happened. | The evidence does not show how common or effective any of the three uses were..

  169. Assessment publishedBy this siteGary Marcus

    LLMs in particular are wildly overrated.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-d25d0b-a1.

  170. Assessment publishedBy this siteGary Marcus

    If AGI arrived immediately, society would very likely not be ready for it, given the near-absence of regulation with real teeth.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-ce4259-a1.

  172. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-bc2c70

    e1 is dated 2024, which is not inside the period described (2024-01-01 to 2024-12-06); it can only be context

  173. Assessment publishedBy this siteGary Marcus

    Hallucination-prone AI may soon be used to target weapons, given OpenAI's collaboration with Anduril.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-ac08be-a1.

  174. Assessment publishedBy this siteGary Marcus

    Even a 5% chance that LLM returns are not diminishing means society should move immediately to prepare for advanced AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-923589-a1.

  175. Assessment publishedBy this siteGary Marcus

    Society needs to prepare for AGI, whenever it arrives.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-90739d-a1.

  176. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-8d3517

    The anthropic review does not accept e4 as contradicts.

  177. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-808ee9

    The final-text check failed: citations_correct.

  178. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-801f0e

    hedged language is not scored, even on a plain reading

  179. Assessment publishedBy this siteGary Marcus

    Society should prepare now for AI becoming much more dangerous than it already is.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-73636f-a1.

  180. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-685f41

    The anthropic screen reads the claim differently.

  181. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-58c990

    The final-text check failed: citations_correct; overclaims: AI features in web search reached over a billion users a month in 2024..

  182. Assessment publishedBy this siteGary Marcus

    As AI gains power, AI skepticism becomes an increasingly important counterweight and should be done well.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-55d0d1-a1.

  183. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-4100fb

    The openai review found the criteria unfair to the speaker's words.

  184. Assessment publishedBy this siteGary Marcus

    Military misuse of LLMs seems imminent, adding to downsides such as covert racism, deepfakes and discrimination that already exist.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-3c7a27-a1.

  185. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-1f990b

    The anthropic screen finds the claim checkable, so it must be graded.

  186. Assessment publishedBy this siteGary Marcus

    AI needs substantive regulation.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-hard-forked-casey-newtons-distorted-1cdc72-a1.

  187. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-133d87

    The openai review found the criteria unfair to the speaker's words.

  188. HeldBy this siteGary Marcus

    Held record marcus-hard-forked-casey-newtons-distorted-034b55

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  190. Assessment publishedBy this siteGary Marcus

    LLMs doing arithmetic still have trouble with abstraction, even with new architectures and training regimes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-happy-groundhog-day-the-ai-edition-f7b173-a2.

  191. Assessment publishedBy this siteGary Marcus

    In the compositionality study he cites, the measures and the model are new, but the stumbling block is the same one neural networks faced in 1993.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-happy-groundhog-day-the-ai-edition-f55e84-a1.

  194. Assessment publishedBy this siteGary Marcus

    AI systems will probably keep failing on endless edge cases for another few years, unless and until people learn from history.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-happy-groundhog-day-the-ai-edition-cbfc5e-a1.

  195. HeldBy this siteGary Marcus

    Held record marcus-happy-groundhog-day-the-ai-edition-b96215

    The final-text check failed: proposition_faithful.

  197. Assessment publishedBy this siteGary Marcus

    Current AI systems work better than older ones but are still superficial: fine for lab demos, often at a loss in the real world.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-happy-groundhog-day-the-ai-edition-33b9a0-a1.

  198. HeldBy this siteGary Marcus

    Held record marcus-happy-1st-birthday-gpt-4-83bd8e

    The openai screen reads the claim differently.

  199. HeldBy this siteGary Marcus

    Held record marcus-happy-1st-birthday-gpt-4-715ec4

    e2 is dated 2024-03, which is not inside the period described (2023-03-16 to 2024-03-16); it can only be context

  200. Assessment publishedBy this siteGary Marcus

    LLM capability may be reaching a plateau around GPT-4 level.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-happy-1st-birthday-gpt-4-6895cb-a1.