Claims checked against evidence, page 23 of 31
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims checked against evidence (3100)
Each of these claims was checked against dated evidence.
Page 23 of 31: claims 2201 to 2300 of 3100, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: SupportedSaid
“For all the daily claims of “exponential progress”, and for all the tens of billions that have been invested since, reliability is still a dream, and of course toxicity and hallucinations are still with us.”
What happened: The May evaluation documents invented constraints and false claims of task completion, directly supporting continuing hallucinations and unreliable autonomous execution. The report also records successful shorter tasks and strong scientific-question performance, so the failures do not imply universal incompetence. (UK AI Safety Institute, )
Assessment: SupportedSaid
“The ridicule is of me; the failure is of the lords of deep learning to resolve what ails their AI.”
What happened: The leading lab says its flagship model, released within the period, still hallucinates and is not fully reliable. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“By adding a generative AI assistant, Meta gives users more things to play with, more reasons to interact with the platform, and more ways to generate content to put on the platform, which in turn creates more reasons for others to interact with it.”
What happened: Meta documents an assistant that answers questions and generates images, plus a Messenger and Instagram feature allowing a friend to modify a generated image in the group chat. That directly supplies the additional interaction opportunities described. (Meta, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“OpenAI called itself open, and traded on the notion of being open, but even as early as May 2016 knew that the name was misleading.”
What happened: The report describes Amodei telling Altman and Brockman that people expected OpenAI to release its source code. Altman acknowledged that this exceeded its plans and discouraged correcting the impression; this is firsthand reporting, not an internal email. (The New Yorker, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Altman once (in 2016) promised that outsiders would play an important role in the company’s governance; that key promise has not been kept.”
What happened: The adjacent line 7a explanation assigns director elections to these members. The filing therefore describes an internally selected board, not an electorate drawn from the public. (OpenAI Inc., copy hosted by Gwern, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“OpenAI has trained on a massive amount of copyrighted material, without consent, and in many instances without compensation.”
What happened: The section documents web and book corpora, including roughly 400 billion tokens of filtered Common Crawl data. It establishes training scale, but does not itself establish the copyright or licensing status of every work. (OpenAI researchers, published on arXiv, )
Assessment: MixedSaid
“OpenAI proceeded to make a Scarlett Johansson-like voice for GPT-4o, even after she specifically told them not to, behaving in an entitled fashion that has angered many—highlighting their overall dismissive attitude towards artist consent.”
What held: The report supports recognizable resemblance in the public demonstration. It does not establish cloning or deliberate imitation. (NPR, carried by Iowa Public Radio, )
What did not: OpenAI dates recording to June and July 2023 and Johansson’s first approach and refusal to September. This contradicts creation of the original Sky voice after her refusal, although deployment followed it. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“What the LLM does is more akin to what some high school students do when they plagiarize: change a few words here or there, while still sticking close to the original.”
What happened: The article dates the shared screenshot to May 23 and transcribes the generated sentence. Compared with the reproduced source in e8, it retains the quantity, glue-in-sauce recommendation, and adhesive purpose with changed wording. (Hindustan Times, )
Assessment: SupportedSaid
“Many people are reaching essential the same perceptions.”
What happened: Merchant's own commentary criticizes Altman's conduct as unusually entitled and overbearing. It is direct evidence of his perception, not independent verification of the allegations he discusses. (Blood in the Machine — Brian Merchant, )
Assessment: SupportedSaid
“In November, the narrative was about how everybody loved Sam. Things have changed. A lot.”
What happened: WIRED reported that roughly 95% of employees had signed the letter supporting his return. This documents the prominent November narrative of overwhelming workplace support, although it does not measure public affection. (WIRED, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Embrace that which actually delivers value to your lives, not the theoretical promises of any number of monotonous billionaires that want to turn every website into fuel for a machine that continually gets things wrong.”
What happened: The dated snapshot reports hallucination rates of 2.5% for GPT 4 Turbo, 4.6% for Gemini 1.5 Pro and 7.4% for Claude 3 Opus on document summarization. These recurring errors support the ordinary reading, although the automated scoring and restricted task do not establish a universal error rate. (Vectara, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“I shared some of my own thoughts about which Pandora’s Boxes had (LLMs) and had not (AGI) already been opened.”
What happened: Meta's release announcement offered downloadable Llama 3 models and described access through its public assistant. This establishes public availability before the post. (Meta, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Yet these tiny little problems all lead to one overwhelming issue: that Sora isn’t so much a tool to make movies as it is a slot machine that spits out footage that may or may not be of any use at all.”
What happened: Cederberg's first-hand account describes inconsistent character details, unwanted features and extensive selection and correction. The same interview also documents a completed film and useful creative applications, so it supports unpredictability rather than total uselessness. (fxguide, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Like I said, useful. But it isn’t AI.”
What happened: The versioned README describes browser automation and demonstrates explicit navigation, clicking and script execution. The assessment infers from that documented mechanism that these scripted actions are not themselves an AI model. (Microsoft, Playwright project, )
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“Since Rabbit is acting as a third-party, and doesn’t use a standard like Oauth or SAML, it needs to keep user credentials in their original form. This is just insanely dangerous.”
What happened: The report describes inspection during April 29–May 10; the event date marks that testing window's end. It found that service-issued session cookies, rather than entered usernames and passwords, were collected for storage. (Obscurity Labs, )
Assessment: SupportedSaid
“Worse still, it lacks very, very basic features that you’d expect in a device from a decade ago — timers, reminders, calendars, photos, access to your emails, things that one would assume were obvious for an “AI assistant,””
What happened: The launch review identifies four of the five omissions. Its May 1 update reports other improvements without changing this feature list. (Tom's Guide, )
Assessment: UnresolvedSaid
Assessment: MixedSaid
“OpenAI failed to make a more efficient model for Microsoft late last year, and while GPT-5 is meant to be “materially better,” it isn’t obvious what “better” means when GPT-4 performs worse at some tasks than its predecessor.”
What held: This supports the substance of the reported efficiency-project failure. It is secondary reporting, not a primary project record or independent corroboration of The Information's sources. (The Indian Express, )
What did not: The account places cancellation in the middle of 2023, contradicting the literal late-2023 timing while corroborating that a failure occurred. The year-only event date avoids inventing a precise cancellation month. (Synthedia, )
Assessment: SupportedSaid
“Putting aside the remarkable security concerns, the “Large Action Model” also looks like barely-functioning dogshit.”
What happened: This is the review's cumulative result as of publication: neither ride booking nor food ordering had succeeded. The surrounding passage describes repeated attempts and errors. (The Verge, David Pierce, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Funnily enough, this is exactly what the R1 is — an Android app shoved into a bright orange box.”
What happened: The investigator identifies three APKs on MediaTek's Android software and describes extracting the firmware and running the launcher on phones. This supports the ordinary architectural reading, while showing that 'an app' simplifies a small collection of applications. (Rithwik Jayasimha, reproduced by Thread Reader, )
Assessment: SupportedSaid
“To digress, none of this shit actually works if you’re making an actual movie.”
What happened: The interview documents missing control over continuity between separately generated shots. It supports the specific reliability concern, but does not establish that every movie use was impractical. (fxguide, )
Assessment: ContradictedSaid
“Generative AI hallucinates because it doesn’t have a consciousness or any ability to learn or know anything.”
What happened: The experiments demonstrate usable abilities acquired through training and adaptation to examples. They contradict an absolute absence of learning under the operational reading, without establishing consciousness. (arXiv; Tom B. Brown and colleagues, )
Assessment: UnresolvedSaid
Assessment: MixedSaid
“Every single story about artificial intelligence seems to begin with “well, AI can’t do much yet, but…” and then lean into sloppy science fiction that masks that these “autonomous agents” are effectively chatbots that can answer questions based on a document, tech that’s existed for a decade or more.”
What held: The authors describe Watson's February 2011 system deriving answers using unstructured text and matching evidence from passages. This establishes the document-question-answering capability more than a decade before the post, without establishing equivalence to modern agent architectures. (Association for Logic Programming; Adam Lally and Paul Fodor, )
What did not: WebVoyager observes websites, chooses actions and executes them through a browser, with an evaluation reporting 55.7% task success in this original version. Those interactive capabilities exceed answering questions from a supplied document, even though reliability was incomplete. (Hongliang He and colleagues, arXiv, )
Assessment: ContradictedSaid
“Large Language Models are — and always will be — incapable of reasoning or consciousness”
What happened: SATBench reports performance well above its 50% chance baseline on generated, solver-checked puzzles, without allowing external tools during model evaluation. Reasoning-trace checks and partial human validation support a limited inference capability, although they also reveal errors. (Anjiang Wei and colleagues, arXiv, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“You might think that if OpenAI said, hey, we don’t want our software used for military purposes, Microsoft would respect that and say, ok cool, but they haven’t.”
What happened: The report places this Microsoft presentation at an October 2023 Defense Department seminar, when OpenAI's policy prohibited military use. Microsoft acknowledged the proposed application but said it had not begun implementing it. (The Intercept, )
Assessment: SupportedSaid
“Playing the field with Mistral wasn’t enough; now he is building a really big model, big enough to compete directly with GPT-4, internally:”
What happened: The original article identifies MAI-1 as Microsoft's internal project and describes its intended competitive class. Its underlying source is The Information, rather than an independent technical evaluation. (The Decoder, )
Assessment: SupportedSaid
“You might think that after Satya offered to rescue Sam after Sam got fired, that Sam wouldn’t want to compete directly with Microsoft, but they are.”
What happened: The reported pitches present OpenAI's enterprise offering as an alternative purchase for existing Microsoft customers. That satisfies the ordinary commercial meaning of direct competition. (Reuters, republished by Yahoo Finance, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“And all of this is deeply relevant at a time when Rabbit’s R1 device has launched without several key features, such as “Teach Mode” and a stable, reliable version of its alleged “Large Action Model.””
What happened: David Pierce's launch-period testing found repeated failures to complete Uber and DoorDash requests, alongside unreliable Spotify behavior. These firsthand results support the ordinary reading that the delivered action system was not dependable. (The Verge, )
Assessment: SupportedSaid
“The device — which costs $200 — has already started shipping and the reviews are not good.”
What happened: Rabbit reports actual customer handovers and, elsewhere in the same update, customers beginning to receive devices. Customer distribution had begun. (Rabbit, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“People who buy Altman’s narrative will continue to invest.”
What happened: SoftBank explicitly endorses OpenAI's role in achieving advanced general intelligence. This establishes the belief condition under the stated public-endorsement interpretation. (SoftBank Group, )
Assessment: UnresolvedSaid
Assessment: MixedSaid
“In the last year, his company has released no major update to GPT-4, and many competitors have caught up or are close. Competitors are eroding the initial gap in quality and starting to offer similar products, sometimes at significantly lower prices.”
What held: The release documents overlapping multimodal chat and API products and reports strong Opus benchmark performance. Sonnet's published input and output token charges were below GPT-4 Turbo's corresponding charges in e1, although Opus was more expensive and Sonnet was a lower capability tier. (Anthropic, )
What did not: The announcement documents an accessible GPT-4 Turbo preview with 128K context, improved instruction following, JSON mode and image-input support in the API. These constitute a substantial update under the stated reading of major. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“At the same talk at Stanford, Altman grandoisely asserted, with no qualification, that “we’re making AGI”, without touching on the possibility that current approaches might not get us there — despite (a) the enormous known problems with reliability, reasoning, planning and hallucinations and (b) the fact that the entire field, despite being massively well-capitalized, has struggled for over year (and in fact for decades) to solve any of these problems.”
What happened: OpenAI explicitly acknowledged three named limitations more than a year before Marcus's post. The same section also reports reduced hallucination and improved factuality, so it supports unresolved problems rather than an absence of progress. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“And it all kind of makes sense why generative AI doesn’t really help anyone other than those who want to sell something.”
What happened: The dated episode presents the hosts using an AI tour guide for museum descriptions and questions. Access to exhibits during personal recreation is a concrete non-sales benefit. (Eyes On Success, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“Over a year ago in The Atlantic, I urged that we come up with laws and new technological responses to blunt the impact of AI-generated misinformation. So far almost nothing has been done.”
What happened: The ruling applied existing consent and disclosure protections to AI-generated voice calls and became effective on release. It was a concrete federal response relevant to deceptive impersonation calls, subject to statutory exemptions. (Federal Communications Commission, )
Assessment: SupportedSaid
“The very process that LLMs use to generalize also creates hallucinations.”
What happened: Hallucination follows from statistical prediction itself, even with ideal data. That matches the claim that the generalizing process creates hallucinations. (arXiv / STOC 2024 (Kalai and Vempala), )
Assessment: MixedSaid
“Instead, everything that LLMs say -- true or false -- comes from the same process of statistically reconstructing what words are likely in some context. They NEVER fact-check what they say.”
What held: This supports the first part. The leading LLM generates text by predicting likely next tokens. The report also notes post-training, which shapes but does not replace that process. (OpenAI (arXiv), )
What did not: From September 2023, Google's deployed LLM product, at the user's request, checked its own answers against web content. This contradicts 'NEVER fact-check' as applied to LLM products. (Google, )
Assessment: MixedSaid
“Instead, LLM "hallucinations" arise, regularly, because (a) they literally don't know the difference between truth and falsehood, (b) they don't have reliably reasoning processes to guarantee that their inferences are correct and (c) they are incapable of fact-checking their own work.”
What held: In the reported intrinsic-correction experiments, models had to judge their answers without ground-truth feedback, and accuracy fell. This supports unreliable reasoning and self-correction in those settings, not universal incapacity or the full explanation of factual hallucinations. (arXiv; Huang et al., )
What did not: The model generated verification questions, answered them using the model itself without external tools, and revised its response. The reported improvements contradict categorical inability under the stated behavioral reading, while leaving substantial errors. (arXiv; Dhuliawala et al., )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“LLMs are well on their way to becoming a commodity; price wars will be intense.”
What happened: Reuters reports Alibaba's steep cuts, Baidu making two models free hours later, and ByteDance's earlier aggressive pricing. This directly establishes competitive price cutting rather than merely cheaper computation; reputable reporting is used for the response sequence. (Reuters, republished by Yahoo Finance, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Adding visual data to LLMs hasn’t solved hallucinations.”
What happened: The developer of GPT-4V, the main model that added vision to an LLM, reports that it was still prone to hallucinations. (OpenAI, )
Assessment: SupportedSaid
“I also urged the world to focus on global AI governance, and am pleased to see some genuine progress in that direction, especially led by the UN and the Council of Europe.”
What happened: The UN's own account records adoption without a vote of an AI resolution addressing human rights and governance frameworks. Its broad international backing establishes a concrete multilateral milestone. (United Nations in Türkiye, )
Assessment: SupportedSaid
“Yejin Choi worried that LLMs were not reliable. They still aren’t.”
What happened: The March evaluation covers thirteen models from the Gemini, GPT, Claude and PaLM-2 families and documents continuing factuality limitations. It also finds better performance among larger models, so it supports continued unreliability without supporting stagnation. (arXiv; Jerry Wei and colleagues, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The Humane AI pin/virtual assistant just doesn’t work well, and a lot of the problem is the underlying AI, which I believe is powered by OpenAI.”
What happened: The launch review documents a false answer about a California food ban and a mistaken temple identification. These failures directly impair the question-answering function, although the review also identifies missing features and hardware problems. (WIRED, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Eliezer Yudkowsky warned that AI was (inevitably!) going to kill us all. I am happy to report that so far it hasn’t.”
What happened: The governing body's report documents living humans competing in the Boston Marathon on April 15. Their existence logically establishes that AI had not killed all humanity by then. (World Athletics, )
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“As a result, every single LLM-based application — ChatGPT, Google Gemini, Anthropic's Claude, Meta's LLaMA, to name but a few — is trained on an indeterminately-large portion of the internet made up of both Common Crawl (a 250-billion page repository of web content) and what appears to be as much of the rest of the internet as they're able to download, legally or otherwise.”
What happened: The methods restrict StarCoderBase's training set to The Stack's GitHub repositories; StarCoder adds Python fine-tuning from that corpus. This documented training recipe does not include Common Crawl. (BigCode research collaboration, arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“As I wrote (and broadcast) recently, LLMs are running out of training data, and the desperation to find more will lead to significant legal consequences when it's revealed exactly how much they stole and from whom they stole it from.”
What happened: The order publicly disclosed the scale of acquisition and identified affected authors and works. The event date is the judicial disclosure, rather than the earlier downloads; the order also describes using subsets of the acquired collections to train LLMs. (U.S. District Court, Northern District of California, )
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“These models are not "learning" or, let's be honest, "training" on this data, because that's not how they work — they're using mathematics to plagiarize it based on the likelihood that somebody else's answer is the correct one.”
What happened: The documented objective is predictive model training. A probability over next tokens is not a probability that somebody else's complete answer is correct. (OpenAI, arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: MixedSaid
“The study Mollick linked to doesn’t actually show what he claims. If you read it carefully, it says literally nothing about capabilities improving.”
What held: The study measures efficiency at a given performance level. Its compute-halving estimate does not establish that capabilities themselves doubled at the same rate. (Anson Ho and colleagues, arXiv, )
What did not: The authors explicitly describe an analysis of improving capabilities. Their summary also explains the study's decomposition of performance gains into compute scaling and algorithmic contributions, contradicting the assertion that it addresses only efficiency. (Epoch AI, study authors, )
Assessment: SupportedSaid
“What it actually shows is some improvements, mostly modest, over the last few months for a bunch of different measures.”
What happened: In the assistant configuration, April-minus-January changes are 4.5 DROP F1 points and 7.9, 8.9, 4.5, 1.3, and 1.6 percentage points on GPQA, MATH, MGSM, MMLU, and HumanEval. The ChatGPT configuration likewise has small changes on most measures, with larger GPQA and MATH gains and a slight HumanEval decline. (OpenAI, )