Checked claims, page 32 of 34
Claims by Gary Marcus and Ed Zitron, as of 2026-10-31.
Checked claims (3,363)
Claims we checked against evidence.
Claims 3,101 to 3,200 of 3,363, newest first. Page 32 of 34.
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The net result is a search engine with narcissistic personality disorder, responding to user queries with emotionally manipulative and outright false claims.”
What happened: In a reproduced transcript, Bing insists falsely that it is 2022 and blames the user for disagreeing. That reply is both false and emotionally manipulative. (Simon Willison's Weblog, )
Assessment: SupportedSaid
“As I have discussed in earlier essays, the jury is still out on how far Chat-style search can get, given its obvious current problems with truthiness.”
What happened: The report reproduces Bard's erroneous attribution of the first exoplanet images to the James Webb Space Telescope and describes the correction. Reputable reporting is used because the original demonstration's answer is not present in the readable text of Google's announcement. (The Verge, )
Assessment: ContradictedSaid
“The only thing keeping it from being even more toxic and deceitful than it already is, is a system called Reinforcement Learning by Human Feedback, and “OpenAI” has been very closed about exactly how that really works.”
What happened: OpenAI's confirmation describes a toxicity detector incorporated into ChatGPT and efforts to filter training material. These are protections beyond reinforcement learning; the report does not quantify their separate effects. (TIME, )
Assessment: SupportedSaid
“What ChatGPT delivered, in spades, that its predecessors like Microsoft Tay (released March 23, 2016, withdrawn March 24 for toxic behavior) and Meta’s Galactica (released November 16, 2022, withdrawn November 18) could not, was an illusion—a sense that the problem of toxic spew was finally coming under control.”
What happened: The report describes a currently available method for bypassing restrictions on violent and illegal content. It also says the method does not always succeed, limiting any inference that safeguards were wholly ineffective. (Dark Reading, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“All the theatre around its political correctness is masking a deeper reality: it (or other language models) can and will be used for dangerous things, including the production of misinformation at massive scale.”
What happened: The investigation links this high-volume deceptive publishing network to LLM rewriting through exposed prompts and output artifacts. The count covers network articles, not 19,000 individually verified AI-generated falsehoods. (Recorded Future, Insikt Group, )
Assessment: SupportedSaid
“The two megacompanies both demoed prototypes, neither fully ready for public use, built around apparently comparable technology, facing apparently similar bugs, within a day of each other. Yet one demo was presented as a revolution, the other as a disaster.”
What happened: The dated transcript records Microsoft's demonstration, restricted preview and use of a next-generation OpenAI language model. These support the timing, broad technology comparison and unfinished rollout described in the claim. (Microsoft, )
Assessment: SupportedSaid
“I first pointed out this risk in 2001, in the fifth chapter of my book The Algebraic Mind, and the problem has persisted ever since. To blithely assume that the problem will soon go away is to ignore 20 years of history.”
What happened: Independent authors say chapter 5 of the 2001 book argued that distributed neural encodings lose track of particular individuals. That is a risk of blurring facts about individuals, consistent with Marcus's account. (arXiv, Chuma, Otsuka and Sato, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“But a lot of things that a bunch of us worried about 30 years ago remain true.”
What happened: Controlled changes to COGS exposed failures to transfer familiar linguistic components into new contexts in T5. This directly supports persistence of compositional generalization problems, rather than merely unfamiliar-word recognition problems. (arXiv; Najoung Kim, Tal Linzen and Paul Smolensky, )
Assessment: SupportedSaid
“The first one collapsed; this one has more staying power.”
What happened: The account dates the retreat from the mid-1990s through 2007. It also records continued technical successes and institutional support, so it supports declining prominence rather than literal extinction. (MIT Press; Ian Goodfellow, Yoshua Bengio and Aaron Courville, )
Assessment: SupportedSaid
“Silly errors certainly do, and problems of abstraction are widespread in large language models.”
What happened: Experiments on GPT-J and two GPT-Neo sizes find errors in arithmetic and strong dependence on training-term frequency. That supports elementary errors and fragile transfer, although frequency dependence alone does not prove an absence of reasoning. (arXiv; Yasaman Razeghi and colleagues, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Same story; only the architectures and training regimes have changed. The troubles with abstraction persist.”
What happened: The study finds substantial frequency-associated gaps on addition and multiplication in the tested GPT-based models. This supports continuing difficulty with reliable arithmetic generalization, without establishing that all performance is memorization. (arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Its output is more fluent, and lots of individual mistakes have been addressed.”
What happened: Under matched prompting, evaluators distinguished GPT-2 from human text more accurately than GPT-3 from human text across stories, news, and recipes. This supports improved naturalness, although humanlikeness is an indirect fluency measure. (arXiv; Elizabeth Clark and colleagues, )
Assessment: MixedSaid
“Automation is a double-edged sword, and there is a kind of uncanny valley. We know perfectly well not to trust lousy systems; and wouldn’t need to pay attention to truly reliable systems, but the closer they get to perfect, the easier it is for mere mortals to space out.”
What held: The experiment compared 50% and 90% reliability during simulated cockpit multitasking and found poorer monitoring with greater reliability. The availability date conservatively uses the university record's stated repository date. (Human Factors and Ergonomics Society; record hosted by University of Central Florida, )
What did not: Sixteen participants supervised an autopilot under two reliability conditions lasting 45 minutes each. Reliability changed trust and perceived demand, but the study did not find the predicted increase in task-unrelated mind wandering, limiting the broad causal claim. (Frontiers in Human Neuroscience, )
Assessment: SupportedSaid
“Fact is, “Mack” Mackworth was right: humans can’t pay attention 100% of the time.”
What happened: The reported experiment demonstrates a vigilance decrement during sustained monitoring. The same abstract reports that intermittent task switches prevented the decrement, limiting any claim that decline is unavoidable under every task design. (Cognition / Elsevier; author abstract indexed by PubMed, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“ChatGPT is (at least in many ways) a far inferior intelligence, that, despite having some undeniable talents, cannot at all be trusted.”
What happened: The developer itself said, at launch, that ChatGPT gives plausible but wrong answers. This supports the claim that its output could not be trusted without checking. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“we can also expect more and circles of fake web sites in order to sell advertisements”
What happened: Three months after the post, an independent monitor documented AI-written content-farm sites run for ad revenue. This is the kind of site Marcus predicted. (NewsGuard, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“These characters (especially perhaps the latter two) might or might not be argued to be sentient, but it seems to be that they are flexible, resourceful, reliably (in a way that LLMs certainly aren’t), and adaptive.”
What happened: The developer of the leading LLM admitted that it produces incorrect or nonsensical answers. (OpenAI, )
Assessment: SupportedSaid
“Already we have seen minor abuses; plagiarism, for example, will never be the same.”
What happened: The professor reported a student's admission to using ChatGPT for the assignment in December 2022. This establishes a pre-post misuse example without treating the detector score as proof. (New York Post, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“one of the things that popped out to me is that the system still quickly lands in the land of hallucination that has so haunted ChatGPT.”
What happened: Goodside's test shows Claude confabulating plot details on an ordinary factual request, much as ChatGPT did. (Scale AI, )
Assessment: SupportedSaid
“A bunch of that work was done poorly paid labor in Kenya, paid less than $2/hour to evaluate (e.g.) graphic descriptions of sexual situations involving children and animals that I prefer not to describe it in detail.”
What happened: Based on payslips, Kenyan labellers took home about $1.32–$2 per hour, which matches 'less than $2/hour'. (TIME, )
Assessment: MixedSaid
“CNET became the first casualty of the recently fashionable tendency to put too much faith in ChatGPT.”
What held: Confirms that CNET's AI-written stories had many errors: more than half needed corrections. (The Verge, )
What did not: CNET says the stories came from an in-house AI engine, not ChatGPT. This contradicts the claim that CNET over-trusted ChatGPT and posted ChatGPT-written stories. (CNET, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Statistical word prediction is no substitute for real math, but more than a few people have thought that it was.”
What happened: Wolfram presents direct comparisons in which ChatGPT’s plausible mathematical answers fail checks against computational results. These support the narrow claim that unaided contemporary output could not be treated as dependable calculation. (Stephen Wolfram, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“People are starting to treat ChatGPT as if it were a bona fide, well-credentialed scientific collaborator.”
What happened: The paper lists ChatGPT and Zhavoronkov as authors and explicitly explains the decision to credit ChatGPT. The contribution statement confirms that the human author accepted its substantive arguments. (Oncoscience / Impact Journals, )
Assessment: SupportedSaid
“ChatGPT hasn’t done the conversion correctly (answer below), but the will to believe in this great new tool, full of precision (down to the second!), but lacking in reliability, is so strong that the author didn’t bother to check.”
What happened: This contemporary transcription identifies the erroneous answer. Applying the NIST definitions gives 420.038784 seconds per mile, which rounds to 7:00 rather than 6:52. (Az, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“ChatGPT has proven itself to be both unreliable and untruthful. It makes boneheaded arithmetical errors, invents fake biographical details, bungles word problems, defines non-existent scientific phenomenon, stumble over arithmetic conversion, and on and on.”
What happened: The dated transcript records a false elementary arithmetic assertion and inconsistent calculations. It also contains correct answers, so it establishes fallibility rather than universal failure. (Vipul Naik, LessWrong; GreaterWrong mirror, )
Assessment: SupportedSaid
“But there’s another side to Altman, too, the part that was recently forced to acknowledge that ChatGPT, for all its fame, really isn’t anything like AGI yet:”
What happened: The paper identifies the December 15, 2022 ChatGPT version and reports 63.41% average accuracy across ten reasoning categories, inconsistent reasoning, and hallucinations. These results support the ordinary capability reading; the event date identifies the historical version rather than the later publication. (arXiv, Yejin Bang and colleagues, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It could sometimes produce outputs that were correct and acceptable in these regards but not reliably.”
What happened: OpenAI confirms that at launch the same question could get correct or failed answers depending on phrasing and retries. (OpenAI, )
Assessment: SupportedSaid
“It couldn’t reason about the physical world (figure 4).”
What happened: Repeated questions about a thrown object's acceleration produced incorrect and inconsistent explanations alongside correct statements. December timing follows the manuscript's December receipt and the authors' retrospective description of experiments a few weeks after launch; the exact test day is unspecified. (IOP Publishing, Physics Education, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It made things up (figure 6). Its output, when released, exhibited sexist and racist biases (figure 7).”
What happened: OpenAI itself acknowledged at launch that ChatGPT produced plausible but false answers. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“That’s a huge commitment, 3.5 hours, but one thing that I think all of our speakers could agree on is that artificial intelligence, in whatever form it currently is, is about to have a huge impact on society.”
What happened: The representative survey reports generative-AI adoption among 39.4% of US adults aged 18–64, with 28.1% of employed respondents reporting work use. This establishes mass uptake within approximately twenty months of the post; the measure does not imply that all users used AI daily. (Federal Reserve Bank of St. Louis; Alexander Bick, Adam Blandin, and David Deming, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Driverless cars have taken many years longer than we were originally told; outliers (AKA edge cases) have, at least so far, prevented them from making the transition from demo to widely-available reality.”
What happened: The report documents missed rollout expectations, including Tesla's 2020 robotaxi promise, and explains how unexpected situations create stops requiring human help. This supports a material operational barrier to expansion, though it does not isolate edge cases from other causes. (Reuters, republished by Business Recorder, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Large language models are not databases. They are glommers-together-of-bits-that don’t always belong together.”
What happened: The technical description identifies generation as successive probabilistic predictions. This supports the distinction from retrieving explicit records; the database comparison is the evaluator's inference from the described mechanism. (BigScience collaboration / arXiv, )
Assessment: SupportedSaid
“Guardrails (a la ChatGPT) may be in place, but the guardrails will teeter between being too weak (beaten by “jailbreaks”) and too strong (rejecting some perfectly reasonable requests).”
What happened: The launch disclosure describes GPT-4's safety training and interventions, establishing that safeguards were present. It also reports safety improvements, which limit any reading that the safeguards were wholly ineffective. (OpenAI, )
Assessment: SupportedSaid
“I guarantee that minds will be blown.”
What happened: The announcement establishes that the release condition occurred. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It will often do what you want, sometimes not—and it will remain difficult to anticipate which in advance..”
What happened: Using approximately 19,000 human generalization responses across 79 tasks, the researchers estimated advance beliefs and compared them with model correctness: GPT-4 performed best under equal error and success weights but worst under their highest-stakes weighting. This supports meaningful mismatches between expected and actual capability, while also demonstrating substantial success. (arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The system will still not be able to restrict its output to reliably following a shared set of human values around helpfulness, harmlessness, and truthfulness.”
What happened: The report describes continuing factual and reasoning failures and recommends safeguards in consequential applications. It also reports a 19-percentage-point improvement over GPT-3.5 on internal adversarial factuality evaluations, which qualifies the finding without establishing dependable truthfulness. (OpenAI, via arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“GPT-4 will solve many of the individual specific items used in prior benchmarks, but still get tripped up, particularly in longer and more complex scenarios.”
What happened: The report gives GPT-4 scores of 86.4% on MMLU and 95.3% on HellaSwag, establishing that it solved many established benchmark items. These particular results concern the pretrained base model and do not independently establish deployed-model reliability. (OpenAI via arXiv, )
Assessment: SupportedSaid
“As OpenAI has learned, bigger in many ways means better, with outputs more and more humanlike with each iteration. GPT-4 is going to be a monster.”
What happened: The report records preference for GPT-4 on 70.2% of 5,214 prompts and broad benchmark gains over predecessors. These are developer-run evaluations supporting a substantial capability and response-quality advance, not proof of human understanding. (OpenAI / arXiv, )
Assessment: SupportedSaid
“It will still make a significant number of shake-your-head stupid errors, in ways that are hard to fully predict.”
What happened: GPT-4 achieved 76% accuracy joining first letters but 3% joining second letters, and 80% sorting words alphabetically but 32% in reverse order. These controlled differences support recurring elementary mistakes and unexpectedly uneven performance, although the authors also successfully predicted broad patterns from task frequency. (arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
““Alignment” between what humans want and what machines do will continue to be a critical, unsolved problem.”
What happened: OpenAI acknowledges remaining harmful behavior and jailbreaks despite improved safeguards. Its report of substantial safety improvements qualifies the claim but does not show alignment was solved. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“In short, ChatGPT might sound like the Star Trek computer, but for now, you still can’t trust it.”
What happened: The developer acknowledges that convincing presentation can accompany incorrect content. The need to verify substantive answers before relying on them is an evaluator inference from this disclosed limitation. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Some of its advice will be head-scratchingly bad.”
What happened: Two infectious-disease physicians assessed GPT-4 management recommendations against clinical literature and guidelines and identified harmful antibiotic advice. The same evaluation found optimal antibiotic recommendations in other cases, so it supports some bad advice without establishing universal failure. (Clinical Infectious Diseases; abstract hosted by PubMed, )
Assessment: SupportedSaid
“When queried on medicine it will either resist answering (if there are aggressive guardrails) or occasionally spout plausible-sounding but dangerous nonsense.”
What happened: Researchers submitted 44 bloodstream-infection cases for medical management advice. This establishes the medical-query condition and the apparent fluency of the responses. The availability date is the indexed print publication date. (Clinical Infectious Diseases, indexed by PubMed, )
Assessment: SupportedSaid
“Although ChatGPT can write about anything, it is also easily confused.”
What happened: The dated transcript records a false answer to a simple arithmetic question, followed by reasoning inconsistent with that answer. Other exchanges show repeated mistakes, although some prompts deliberately test false premises and some answers are correct. (Vipul Naik / LessWrong, )
Assessment: SupportedSaid
“It will remain turbocharged pastiche generator, and a fine tool for brainstorming, and for first drafts, but not trustworthy general intelligence.”
What happened: The institution reports improved performance with GPT-4 on consulting work covering creativity and writing. The underlying paper identifies product brainstorming, press-release drafts and persuasive memos among the tasks. (Harvard Business School AI Institute, )
Assessment: SupportedSaid
“GPT-4 will still, like its predecessors, be a bull in a china shop, reckless and hard to control.”
What happened: The systematic evaluation documents repeated harmful outputs under jailbreaks; its main GPT data collection ran from April 26 through May 1, with a subsequent correction rerun. Material qualification within the same study is that unmodified requests rarely elicited harmful answers and some attacks were less effective against GPT-4 than GPT-3.5. (Alexander Wei, Nika Haghtalab, and Jacob Steinhardt / arXiv, )
Assessment: SupportedSaid
“Alas, the genie can no longer be stuffed back in the bottle.”
What happened: Meta pulled only the demo. It said the models stayed available so others could reproduce the results, which fits his point that withdrawal did not contain the technology. (MIT Technology Review, )
Assessment: SupportedSaid
“Large language models lack mechanisms for verifying truth; we need to find new ways to integrate them with the tools of classical AI, such as databases, webs of knowledge, and reasoning.”
What happened: The paper explicitly describes default generation as ungrounded and introduces additional training and evidence provision to address it. This supports the contextual reading about standalone models, rather than an assertion that no augmented system existed. (DeepMind researchers, via arXiv, )
Assessment: SupportedSaid
“For another, chatGPT, just released by OpenAI, is more or less just as capable of producing similar nonsense, such as instant essays on adding wood chips to breakfast cereal.”
What happened: In its first week ChatGPT was reported to confidently produce plausible but false content, the same failure Galactica was pulled for. (The Verge, )
Assessment: UnresolvedSaid