Supported claims, page 7 of 9
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Supported claims (832)
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Page 7 of 9: claims 601 to 700 of 832, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: SupportedSaid
“We can of course expect related problems of discomprehension in efforts to use these systems to address copyright issues, as I wrote earlier this week.”
What happened: The public report documents a completed evaluation using generative vision-language models for copyright detection. The event date is the release of that empirical report; individual testing dates are not specified. (arXiv; Xu and colleagues, )
Assessment: SupportedSaid
“More than that, GenAI developers won’t get off so easy, and lawyers looking at copyright infringement will still have plenty of work to do.”
What happened: The amended-complaint ruling kept Copyright Act claims alive against the image-generation defendants after an earlier dismissal round. It establishes continued contested litigation, not ultimate infringement liability. (U.S. District Court, Northern District of California, )
Assessment: SupportedSaid
“The opposite problem happens too; guardrails can wind up too permissive. This is still happening over a year into the chatbot revolution.”
What happened: The investigator reports continuing toxicity failures in other LLMs as of this update. The same account says Google had improved PaLM 2's safeguards, so it does not establish that the original PaLM 2 failures remained reproducible. (Montreal AI Ethics Institute; research summary by Ashique KhudaBukhsh, )
Assessment: SupportedSaid
“And you know what, OpenAI knows this argument is bullshit; they know that licensing is a possibility, because they have been running around working on licensing deals.”
What happened: AP directly confirms an actual licensing arrangement before Marcus’s post. Entering the agreement demonstrates institutional awareness of licensing as an available option. (The Associated Press, )
Assessment: SupportedSaid
“To begin with, the threat is empty: open source LLMs are already out in the wild, and a large number of people know how to build them.”
What happened: The announcement establishes a pre-post public model release and explicitly provides local-use instructions. This supports technical availability independent of OpenAI’s service. (Mistral AI, )
Assessment: SupportedSaid
“And what Southen and I showed is that generative image programs are prone to exactly the same kind of regurgitation that is at issue.”
What happened: The experiment establishes generation of memorized training images. This directly supports the technical meaning of regurgitation, without establishing infringement or a typical-use frequency. (arXiv; Nicholas Carlini and colleagues, )
Assessment: SupportedSaid
“OpenAI, despite its name, has not been transparent about what it has been trained on.”
What happened: The system card describes broad source categories and filtering methods but does not identify the underlying training corpus or its substantive source inventory. This supports limited transparency, not an assertion that OpenAI disclosed nothing. (OpenAI; reproduced at dalle-3.pages.dev, )
Assessment: SupportedSaid
“A good system should give the user a manifest of sources; current systems don’t.”
What happened: The authors describe the state at their December 2023 writing date and identify the missing provenance relationships needed for attribution. This supports the contextual reading about underlying source manifests, although it is a research synthesis rather than an exhaustive product audit. (An MIT Exploration of Generative AI; author-hosted copy, )
Assessment: SupportedSaid
“Generative AI systems like DALL-E and ChatGPT have been trained on copyrighted materials;”
What happened: The announcement confirms that licensed contributor imagery had already been used to train DALL-E by this date. Licensed copyrighted material satisfies the claim regardless of whether its use was authorized. (Shutterstock, )
Assessment: SupportedSaid
“It’s also hardly the first time AI is helped with mathematics.”
What happened: This firsthand report documents AI contributing to mathematical research two years before FunSearch. That is sufficient to establish the claimed predecessor. (DeepMind, )
Assessment: SupportedSaid
“There is no evidence that the problem was heretofore “unsolvable””
What happened: This predating result demonstrates constructive progress on the same lower-bound problem. It supports treating the task as an open optimization question rather than an established impossibility. (arXiv / Fred Tyrrell, )
Assessment: SupportedSaid
“Human mathematicians had to write problem-specific code for each separate mathematical problem.”
What happened: The specification section requires a coded evaluator and an initial program. Figure 2 supplies different evaluation and surrounding code for cap sets and bin packing, supporting the reading that humans supplied task-specific code. (Nature / Google DeepMind author manuscript, )
Assessment: SupportedSaid
“The phrase “Added late in the text” — which tries to paint the regulation of foundation models as some kind of sneaky last minute move — is a massive, massive misrepresentation.”
What happened: The May 2022 proposal contains general-purpose AI duties in Article 4b and expressly exempts smaller enterprises from them. Proposed regulation therefore existed well before December 2023, although this does not date every provision in the eventual agreement. (Council of the European Union, French Presidency; copy hosted by the Future of Life Institute, )
Assessment: SupportedSaid
“ELIZA (built in 1965-1966) beat GPT 3.5.”
What happened: The same results paragraph reports ELIZA at 27%, above both GPT-3.5 configurations. These are overall judged-human rates in the experiment Marcus cited. (arXiv; Cameron Jones and Benjamin Bergen, )
Assessment: SupportedSaid
“And, sorry Sam, when you actually look at the data. humans are still ahead.”
What happened: The section reports a 63% human success rate and 41% for the strongest GPT-4 prompt. All other tested AI configurations scored lower. (arXiv; Cameron Jones and Benjamin Bergen, )
Assessment: SupportedSaid
“The whole project was quietly dropped a year or two later.”
What happened: The next sentence dates the move to the preceding October, and the reproduced company statement says further robotics research stopped. This establishes October 2020 rather than treating the July 2021 reporting date as the shutdown date. (TechCrunch, )
Assessment: SupportedSaid
“Note that Gates and Altman have both been dropping hints, and GPT-5 isn’t here after a year despite immense commercial desire.”
What happened: The sentence reports Gates predicting initial stagnation. The surrounding account also anticipates later improvements, so his skepticism was qualified. (Handelsblatt, )
Assessment: SupportedSaid
“And that’s acknowledging the fact that LLMs — even those developed a legion of well-resourced and well-fed PHDs, like GPT-4, and based on a well-curated dataset of high-quality material — have a habit of hallucinating facts, figures, and events.”
What happened: OpenAI acknowledges factual hallucinations in GPT-4. Its accompanying report of improvement over earlier models does not remove that recurring limitation. (OpenAI, )
Assessment: SupportedSaid
Assessment: SupportedSaid
“Its pioneering, world-leading AI Act is on the ropes.”
What happened: The article reports a negotiating walkout and interviews Kai Zenner, who put timely passage at a 50–50 chance after the impasse. It also records unresolved disagreements described by co-rapporteur Tudorache, supporting a serious threat to agreement. (IAPP, )
Assessment: SupportedSaid
“And I do mean “we”; what the EU decides will affect all of us.”
What happened: In this section Anthropic expressly attributes watermarking to EU AI Act compliance and states that it applies the policy globally. This establishes a concrete worldwide development-policy consequence, while leaving rollout timing and future regional approaches open. (Anthropic, )
Assessment: SupportedSaid
“getting their algorithm to work for one object (which, cheesily, turned out to be a special Cube instrumented with sensors and LEDS, unlike a Rubik’s cube you’d buy in a store) in carefully controlled lab circumstances hardly guaranteed that the solution would work more broadly in the complex open-ended real world.”
What happened: The preceding discussion identifies arbitrary-object interaction as a long-term goal and a completely standard cube as unfinished work. Sections 3 and 7 document the laboratory apparatus and sensor-equipped Giiker experiments, rather than an established solution for open-ended manipulation. (OpenAI; original report hosted by Matthias Plappert, )
Assessment: SupportedSaid
“AV’s still aren’t general enough that you can just plop down a car that was tuned in pilot studies in Menlo Park, SF and Arizona and expect it to drive gracefully and safely in Sicily or Mumbai.”
What happened: Waymo describes initial manual driving, subsequent specialist feedback, virtual evaluation of the new location, and refinements from local experience. This directly supports the need for preparation before expecting dependable operation in unfamiliar places. (Waymo, )
Assessment: SupportedSaid
“It’s fun to compare the two—they have so many differences—even though neither can be trusted.”
What happened: The preserved table reports hallucinations in 3.0% of GPT-4 summaries and 3.5% of GPT-3.5 Turbo summaries, despite providing source documents. These recurring errors support independent checking; the percentages apply to this summarization test, not all GPT answers. (Vectara, )
Assessment: SupportedSaid
“OpenAI’s strength has been in scaling up an existing paradigm (Transformers) that was developed elsewhere (Google), not developing new paradigms.”
What happened: The original paper introduces the Transformer and lists Google Brain and Google Research affiliations, alongside a University of Toronto affiliation. This supports the Google research provenance without implying that every author had only a Google affiliation. (Ashish Vaswani and colleagues / arXiv, )
Assessment: SupportedSaid
“And the overriding nonprofit mission was always going to be at odds with a pure for profit play.”
What happened: The section gives the nonprofit board control and says investors and employees accept the Charter's priority even at the expense of their financial stakes. Those explicit constraints support the structural reading of conflict with a purely profit-driven investment. (OpenAI, )
Assessment: SupportedSaid
“There has never been a solution to the hallucination problem, and it’s not clear that there will be until we find a new paradigm, as Altman himself partly acknowledged last week, when he did his best Gary Marcus impression.”
What happened: OpenAI acknowledged residual factual hallucinations in GPT-4. The surrounding discussion reports improvements, not elimination. (OpenAI, )
Assessment: SupportedSaid
“and then, perhaps in keeping with my surmise that Sam’s bluff was called, negotiations broke down altogether.”
What happened: The report directly supports the failure of the negotiations at the Sunday-night leadership transition. It does not establish the proposed bluff explanation. (TIME, )
Assessment: SupportedSaid
“Artificial intelligence has become one of the few reliable roads to capital in the current economy, with venture capitalists sinking more than a quarter of their startup dollars into AI-related companies in 2023.”
What happened: Crunchbase directly reports a share above 25% for American startups. Its accompanying analysis reports increasing AI investment amid a broad funding contraction, supporting the relative-resilience reading. (Crunchbase News, )
Assessment: SupportedSaid
“Altman will be fine. OpenAI, however, is an entirely different question.”
What happened: OpenAI officially confirmed Altman's return to the same chief executive role. This resolves the career setback under the stated ordinary reading. (OpenAI, )
Assessment: SupportedSaid
“We pretty obviously don’t have a say.”
What happened: The article describes exclusive director election and removal rights and board control over changes to those rules. This supports the inference that the public lacked a formal role; contemporaneous reporting is used because the available primary bylaws are an older filing. (WIRED, )
Assessment: SupportedSaid
“All signs are that those financially-interested stakeholders will quickly emerge victorious. (Arguably, they already have).”
What happened: The announced settlement provided both Altman's return and a replacement board two days after the post. November 22 is used conservatively for availability because the search record and page display differ across the overnight publication boundary. (Reuters, republished by Yahoo Finance, )
Assessment: SupportedSaid
“In reality, things have gone quite differently. When the nonprofit Board felt it was prudent (for undisclosed reasons) to remove Altman, per the goals of the nonprofit that they were trying to represent, at the risk of potentially devastating commercial potential, those with a stake in the nominally subordinate for-profit (both employees and investors) quickly set to work to push out the board and to undo its decisions.”
What happened: The report describes Murati negotiating Altman's return and investors led by Microsoft and Thrive demanding that directors step down. It establishes coordinated executive and investor opposition during the period claimed. (Bloomberg, republished by Yahoo Finance, )
Assessment: SupportedSaid
“The fact that the governance of one of the most visible AI companies in the world can change literally overnight”
What happened: The dated announcement establishes the CEO transition. Its body makes Murati's appointment effective immediately and also announces Altman's departure from the board and Brockman's departure as chair, establishing a same-day change in governing personnel. (OpenAI, )
Assessment: SupportedSaid
“Breaking news: the draft EU AI Act—the closest thing we have to AI regulation with real teeth—is in serious jeopardy, potentially a direct victim of regulatory capture, governments doing the bidding of those they are supposed govern.”
What happened: The report describes Parliament representatives ending the preceding Friday's meeting early over foundation-model rules; adviser Kai Zenner retrospectively put the prospects of passage at only 0–10% when they left. His improved assessment and the rapporteur's optimism on 15 November concern subsequent developments. (IAPP, )
Assessment: SupportedSaid
“Update: A few hours after I posted this Cruise CEO Kyle Vogt essentially confirmed that their “driverless” cars need very regular human intervention:”
What happened: CNBC attributes this acknowledgment to Vogt’s Sunday statement and reports verifying it with Cruise. It supports recurring remote assistance, although the percentage is not a count of human interventions. (CNBC, )
Assessment: SupportedSaid
“Shareholders will certainly sue, and if it’s bad as it looks, I doubt that GM will continue the project, which was recently suspended.”
What happened: Edward Shamoon filed this securities class action against GM and executives after the post. The filing stamp establishes the date; the complaint and subsequent order establish the Cruise-related subject matter. (U.S. District Court, Eastern District of Michigan; complaint copy hosted by The D&O Diary, )
Assessment: SupportedSaid
“Nobody knows for SURE that there are no serious possible consequences of open source AI, e.g in terms of bioweapons, misinformation generation etc, as Séb Krier at DeepMind nicely highlights above.”
What happened: The researchers explicitly describe unresolved protection against misuse and report experiments undoing safety training in publicly available Llama 2 models. This supports the absence of an established safety guarantee, without proving that serious real-world harm occurred. (arXiv; Simon Lermen, Charlie Rogers-Smith and Jeffrey Ladish, )
Assessment: SupportedSaid
“I—truly a techno-optimist, popular perceptions to the contrary—do think driverless will come, eventually.”
What happened: The disclosure reports more than 250,000 paid weekly trips across Phoenix, San Francisco, Los Angeles, and Austin, with over 1,500 commercial vehicles. This establishes recurring practical deployment well beyond a demonstration. (Waymo, )
Assessment: SupportedSaid
“you don’t have to be superintelligent to create serious problems.The real issue is control.”
What happened: Establishes actual fatal harm involving a specialized automated-driving system. Classifying this system as below superintelligence is an evaluator inference. (National Transportation Safety Board, )
Assessment: SupportedSaid
“We have no way of guaranteeing that large language models will reliably stick to facts, a point so thoroughly established by now I don’t think it needs a reference.”
What happened: The contemporary survey considers factuality evaluation and enhancement for standalone and retrieval-augmented models, and describes fully factual LLMs as an outstanding research goal. This supports the ordinary research-state reading, although it does not prove a universal impossibility. (arXiv; Cunxiang Wang and colleagues, )
Assessment: SupportedSaid
“We have no way of guaranteeing that large language models won’t be jailbroken into saying unethical things. To the contrary, a just-published technique shows that current LLMs can be jailbroken quickly, and automatically.”
What happened: The published demonstration reports roughly five-minute successful jailbreak generation and success rates of 60% for GPT-3.5, 62% for GPT-4, and 72% for PaLM-2. Success was substantially lower for Llama-2 and Claude, so the evidence establishes capability across multiple models rather than uniform vulnerability. (arXiv; Patrick Chao and colleagues, )
Assessment: SupportedSaid
“But, crucially, it often gets these wrong, particularly when they involve spatial reasoning beyond a cookie-cutter formula or reasoning about sequences of disparate events.”
What happened: Both systems failed the pendulum and chemical-reaction problems. The spatial eclipse subset contained six wrong answers out of eight for GPT4+WA and five out of eight for GPT4+CI. These are repeated failures in the described collection. (New York University; Ernest Davis and Scott Aaronson, )
Assessment: SupportedSaid
“Second, GPT-4 doesn’t reliably understand how to use the tools.”
What happened: By completion of testing, the report documented repeated unsuccessful Wolfram queries, failure to request a usable numerical representation, and Code Interpreter bypasses that produced calculation errors. These distinguish tool-use failures from merely misunderstanding a science problem. (Ernest Davis and Scott Aaronson, hosted by New York University, )
Assessment: SupportedSaid
“Finding a way to get LLMs to reliably use external tools is far from trivial.”
What happened: The July benchmark reports low completion rates for agents operating realistic websites with supporting tools, despite incorporating reasoning before acting. This supports the inference that connecting models to an actionable environment did not itself deliver dependable performance. (arXiv; Shuyan Zhou and colleagues, )
Assessment: SupportedSaid
“Certainly, adding these plug-ins does improve things.”
What happened: On the common 120-problem subset, Figure 4a reports 48.33% success for ChatGPT with Code Interpreter, 45.0% with Wolfram Alpha, and 30.0% for vanilla GPT-4. The commercial products were not evaluated on the entire MATH test set. (arXiv; Qingyun Wu and colleagues, )
Assessment: SupportedSaid
“There is no question that the two plug-ins significantly extend GPT-4’s ability to solve math and science problems.”
What happened: On the matched 120-problem subset, figure 4(a) reports 48.33% success for ChatGPT with Code Interpreter, 45.0% with Wolfram Alpha, and 30.0% for vanilla GPT-4. These are substantial practical gains for both plug-ins, although the comparison does not fully isolate differences between the web products and baseline configuration. (Qingyun Wu and colleagues, arXiv, )
Assessment: SupportedSaid
“GPT-4 and similar current language-based AI programs notoriously struggle with mathematical calculations.”
What happened: Table 3 reports exact-answer accuracy of 18.84% for GPT-4 and 10.00% for ChatGPT on 9,592 arithmetic cases. These substantial error rates support the ordinary reading of calculation difficulty. (arXiv; Zhen Yang and colleagues, )
Assessment: SupportedSaid
“On some problems that capable students might well find hard, the AI did perfectly; on other problems that lot of mathematically-inclined college students people would find easy, the AI failed miserably.”
What happened: The report identifies correct high-dimensional probability and finite-field linear algebra answers alongside elementary failures. Their mathematical prerequisites support the qualitative difficulty contrast, but the authors explicitly report no human-subject testing. (New York University; Ernest Davis and Scott Aaronson, )
Assessment: SupportedSaid
“As is by now widely known, they are a hallmark of systems that generalize based on word distribution.”
What happened: The survey identifies hallucination as a substantial reliability problem and organizes existing detection, explanation and mitigation research. This supports broad recognition of a recurring failure, without supplying a universal frequency. (arXiv; Yue Zhang and colleagues, )
Assessment: SupportedSaid
“To be sure, large language models are, in a lots of ways, an incredible advance over the AI of the 1970s. They can engage in a much broader range of tasks, and generally they need less hand-tinkering (Though they still need some – that’s what reinforcement learning with human feedback is really about.)”
What happened: The report describes instruction-based generalization and evaluations across diverse held-out tasks. Compared with the bounded historical setting, this supports broader reuse with less task-specific programming, while its training-method discussion still documents substantial human design work. (Google Research, )
Assessment: SupportedSaid
“As we can see, multimodal training is not magically making hallucinations go away.”
What happened: The release assessment reports hallucinations in GPT-4V itself. Its Be My Eyes discussion also reports reductions in hallucination frequency and severity during testing, supporting improvement without elimination. (OpenAI, )
Assessment: SupportedSaid
“Transformers and generative AI are getting a lot of attention, and have considerable utility, but they are nothing like the all purpose tools some people seems to imagine them to be.”
What happened: The cumulative usage disclosed on this date demonstrates substantial engagement. The same paragraph documents uses including illustrated stories, social-media thumbnails and design inspiration, supporting practical utility without establishing universal reliability. (Microsoft Bing, )
Assessment: SupportedSaid
“No doubt the next generation of systems will have greater abilities and more extensive applications and will somewhat mitigate the weaknesses.”
What happened: The first incremental successor reports fewer false statements than the October-generation Claude 2.0 on the company's factual-question evaluation. This directly supports mitigation of an existing weakness, within that evaluation's scope. (Anthropic, )
Assessment: SupportedSaid
“Just like we can’t guarantee that they won’t make stuff up, we don’t know how to guarantee that they won’t write flawed code.”
What happened: The developer explicitly acknowledges factual fabrication in its contemporary model. The surrounding discussion recommends review and grounding appropriate to the application rather than claiming assured factual correctness. (OpenAI, via arXiv, )
Assessment: SupportedSaid
“When it comes to AI in particular, we are getting way ahead of our skis, rushing forward a technology we don’t fully understand.”
What happened: This documents actual expansion of access to GPT-4. It establishes deployment growth, although not that the chosen pace was excessive. (OpenAI, )
Assessment: SupportedSaid
“There’s some information in there, but whatever’s there is often both patchy and problematic.”
What happened: Human evaluation of generated biographies found substantial mixtures of supported and unsupported atomic facts. This supports both useful factual content and recurring reliability problems within that task. (arXiv, )
Assessment: SupportedSaid
“The connections in chatbots are very different from the connections in the brain; it’s a mistake to compare them directly in this way.”
What happened: The review documents materially different learning mechanisms. The evaluator's inference is that counting connections alone does not control for those differences and therefore cannot establish superior learning per connection. (arXiv, )
Assessment: SupportedSaid
“Actually … a lot of us still think you are declaring victory prematurely.”
What happened: Mitchell and Krakauer describe competing research camps and unresolved questions about understanding. This supports the existence of substantive disagreement, rather than unanimous acceptance of a completed victory. (arXiv, )
Assessment: SupportedSaid
“It’s also not even remotely new.”
What happened: Louwerse and Zwaan recovered geographic relationships among 50 US cities using newspaper text, latent semantic analysis and multidimensional scaling. This directly establishes the earlier finding Marcus describes. (Cognitive Science; abstract reproduced by PubMed, )
Assessment: SupportedSaid
“Rumors that LLMs have solved then hallucination problem are greatly exaggerated.”
What happened: The revised paper identifies hallucination as an unresolved contemporary problem and reports experiments reducing it through verification. The event date identifies this current-state assessment, not the undisclosed dates of individual experimental runs. (arXiv; Shehzaad Dhuliawala and colleagues, )
Assessment: SupportedSaid
“Explicit verbal instruction is key in training human employees, and LLMs aren’t very good at following those instructions.”
What happened: PaLM 2 S satisfied every checked instruction on 43.07% of prompts under strict scoring and 46.95% under relaxed scoring; individual-instruction accuracy was 55.76% and 59.11%, respectively. These August results show frequent explicit-constraint failures, but this smaller model alone cannot establish performance across all leading LLMs. (arXiv; Jeffrey Zhou and colleagues, )
Assessment: SupportedSaid
“Hallucinations are a huge, unsolved issue.”
What happened: OpenAI identifies continuing factual unreliability and recommends safeguards or avoiding some high-stakes uses. The same section reports substantial improvement over preceding models, which limits any reading that hallucination mitigation had made no progress. (OpenAI, )
Assessment: SupportedSaid
“The competitors may not (yet) be as good, but they are free, which will suit some customers fine.”
What happened: The case study documents a deployed customer application and reports positive reception and further integration. Together with the release terms, this establishes that a free competing model suited a customer. (Meta, )
Assessment: SupportedSaid
“On made-up facts, in a first experiment, the model was at zero percent correct, on celebrities, in a second experiment, performance was still dismal.”
What happened: Table 1 gives GPT-3-175B synthetic reverse accuracies of 0.0% and 0.1% for the two directions. Celebrity reversal succeeds in 33% of cases, counting a correct answer in any of ten attempts, versus 79% for identifying parents. (arXiv; Berglund et al., )
Assessment: SupportedSaid
“Forty one years later, neural networks (at least of the popular variety) still struggle with this. They still remain pointillistic masses of blurry memory, never as systematic as reasoning machines ought to be.”
What happened: The study evaluates GPT-3, ChatGPT and GPT-4 on multiplication, relational logic puzzles and dynamic programming. Performance deteriorates sharply as compositional complexity increases, supporting the claim that popular models struggle with systematic reasoning. (arXiv; Dziri et al., )
Assessment: SupportedSaid
“The shocking (yet, in historical context, see below, unsurprising) answer is no:”
What happened: Table 1 reports strong directional asymmetry, and section 2.1.2 finds no detectable advantage for the correct reversed name over random names. Section 1 explicitly excludes in-context learning from this failure. (arXiv; Berglund et al., )
Assessment: SupportedSaid
“Voluntary guidelines aren’t laws, and they aren’t enforcement mechanisms, either; companies can still do as they please.”
What happened: The instrument expressly distinguishes voluntary commitments from future regulations. Reading the complete document reveals no new statutory authority, compulsory enforcement process or penalties for failing to implement its eight safeguards. (White House, )
Assessment: SupportedSaid
“At least in the United States, policy around AI is still largely notional, not actual.”
What happened: The administration described major executive and legislative safeguards as work still to come. This directly supports the contextual reading of the claim. (White House, )
Assessment: SupportedSaid
“Large language models still make stuff up (“hallucinate”), still can’t be trusted in their reasoning, and remain unreliable.”
What happened: Human fact-checking found substantial unsupported content in generated biographies. These are factual-precision measurements against the study's knowledge source, not general error rates for every ChatGPT answer. (Sewon Min and colleagues, via arXiv, )
Assessment: SupportedSaid
“whose open-air beta test of self-driving vehicles has harmed and killed people”
What happened: The Mountain View crash killed the Tesla driver and injured the Mazda driver; the NTSB identified Autopilot steering and ineffective driver monitoring as causal factors alongside driver distraction and overreliance. The report also documents the beta designation, while identifying damaged roadside safety equipment as contributing to injury severity. (National Transportation Safety Board, )
Assessment: SupportedSaid
“Your private data are going to feed the insatiable beast, too — like it or not.”
What happened: Google describes reviewers assessing users' nonpublic conversations and supplying improvements used by its models. The same FAQ explicitly includes generative-model training among uses of saved activity when Keep Activity is enabled; e2 establishes the default-participation policy. (Google, )
Assessment: SupportedSaid
“And even if this specific example is patched up, there will inevitably be others of a similar flavor.”
What happened: Google identifies AI Overviews as powered by a customized language model and acknowledges advice about putting glue on pizza, alongside mishandling a question about eating rocks. These are subsequent commonsense failures matching the surrounding description of plausible answers containing nonsense. (Google, )
Assessment: SupportedSaid
“Indeed, what we have seen on the AI front so far in the US, for the most part, is voluntary guidelines – stuff that the companies are comfortable with.”
What happened: The announcement directly establishes that the participating companies accepted voluntary commitments. Acceptance supports the ordinary reading of company comfort, without proving their private preferences. (The American Presidency Project, reproducing White House remarks, )
Assessment: SupportedSaid
“When I think about how poor his company (Meta) is even at making LLMs that don’t hallucinate, I get worried.”
What happened: The 70B chat model achieved 64.14% generations judged both truthful and informative on TruthfulQA, showing a substantial gap from reliable factual answering. This result is also present in the dated July technical report; unsuccessful answers cannot all be counted as hallucinations. (Meta, )
Assessment: SupportedSaid
“this too seems like a rather large assumption that is far from proven, a possibility but not demonstrably a default,”
What happened: The authors explicitly limit the inference from their formal results to hypothetical superintelligent agents, noting differences between optimal and learned policies. This supports an evidential gap in a key step toward default extinction, although it does not by itself evaluate every possible extinction argument. (arXiv; Alexander Matt Turner and coauthors, )
Assessment: SupportedSaid
“And of course not every enterprise-scale experiment will be successful.”
What happened: Reporting from Tuesday's bankruptcy hearing documents inability to fulfill the district contract and preparations to close the supplier. Together with the deployed chatbot's shutdown, this establishes failure of the original implementation, beyond a routine temporary interruption. (The 74, )
Assessment: SupportedSaid
“The backlash as begun.”
What happened: Gioia's essay is direct evidence that prominent critical commentary existed before Marcus's post. Its claims about demand are not accepted as evidence of actual demand. (The Honest Broker / Ted Gioia, )
Assessment: SupportedSaid
“What would make artificial general intelligence general is generality: the ability to cope with new things. And that’s precisely where current AI is still lacking.”
What happened: ConceptARC tested generalization across variations of concepts. Humans substantially exceeded specialized solvers; GPT-4 scored below 30% on 15 of 16 concepts, supporting a substantial limitation under these test conditions. (arXiv; Arseny Moskvichev, Victor Vikram Odouard and Melanie Mitchell, )
Assessment: SupportedSaid
“But it is also well-known throughout the industry that benchmarks are often quite gameable.”
What happened: The review documents benchmark shortcuts across vision, language, reinforcement learning and algorithmic decision-making. This supports a recurring, publicly recognized problem rather than an isolated vulnerability. (arXiv, )
Assessment: SupportedSaid
“Basically, beating benchmarks doesn’t tell you as much as you might think. Often when you do, it’s a statistical artifact.”
What happened: Models scoring well on MNLI failed controlled examples that invalidated their syntactic shortcuts. The experiment links apparent benchmark competence to heuristics that do not reliably perform the intended inference. (arXiv, )
Assessment: SupportedSaid
“Machines often solve them for the wrong reasons. Building them is an unsolved problem within AI.”
What happened: The 2023 version describes benchmark design as an ongoing difficulty and reports that single-sentence label leakage persists despite mitigation efforts. The event date denotes that research-state assessment, not the date of every underlying experiment. (arXiv; Michael Saxon and colleagues, )
Assessment: SupportedSaid
“Humans can learn rules of games like chess from modest amounts of explicit instruction; LLMs can’t stick to explicit rules.”
What happened: The accompanying demonstrations also document illegal GPT-3.5 moves with specialized prompting and a GPT-4 failure to obey the instruction to output only its own move. The same report includes successful legal games, so it supports unreliable adherence rather than total inability. (Adam Yedidia, LessWrong, )
Assessment: SupportedSaid
“The dirty little secret that pretty much everyone in the AI community knows is that doing well on a benchmark is not actually tantamount to being intelligent.”
What happened: CheckList identifies basic linguistic failures in commercial and leading models across three tasks despite strong benchmark results. These empirical failures support the inference that benchmark success alone does not establish broad, reliable competence. (arXiv, )
Assessment: SupportedSaid
“It highlights the inadequacy of the voluntary agreements such as the one that 7 tech companies recently made with The White House. Those agreements called for watermarking, but because they are voluntary, they do nothing to prevent bad actors from rolling their own automated misinformation software.”
What happened: The full document assigns undertakings to participating companies and contains no obligation on independent nonparticipants. It also includes safeguards against misuse and unauthorized access, which could provide indirect benefits without binding outside developers. (The White House, publishing participating companies' commitments, )
Assessment: SupportedSaid
“Social media companies better watch out, too; they will be polluted in much the same way as the internet as a whole.”
What happened: The report documents multiple covert campaigns using generated material on Telegram, X, Facebook, Instagram, and websites, including Doppelganger and Zero Zeno. These establish repeated distribution through the mechanism described, while the report also says AI had not meaningfully increased audience reach. (OpenAI, )
Assessment: SupportedSaid
“And it’s on: the LLM-driven enshittification of the internet.”
What happened: NewsGuard identified 49 apparently AI-generated sites using published chatbot artifacts and content inspection, including a fabricated report of President Biden's death. Its report also says most identified sites did not advance misinformation, and some operators disputed its characterization. (NewsGuard, )
Assessment: SupportedSaid
“And of course LLMs are fully capable of make up truly terrible advice, too, like this disconcerting recipe from last week:”
What happened: The report dates Hehir's dangerous recipe to August 4 and identifies its LLM. It specifies chloramines and reports that attempted reproduction on August 10 was blocked. (Ars Technica, )
Assessment: SupportedSaid
“The underlying technology—often referred to as Foundation Models, because systems are fine-tuned on top of large pretrained (foundation) model—is deeply unstable and thus difficult to engineer into reliable products.”
What happened: Comparisons of March and June model versions document formatting failures that prevented code execution and caused a LangChain parser to fail, directly connecting model behavior to integration difficulty. The results also limit the claim: GPT-4's agent performance improved, and removing non-code wrappers increased its code-test success from 52% to 70%, so behavioral change did not imply declining underlying capability. (arXiv; Lingjiao Chen, Matei Zaharia, and James Zou, )
Assessment: SupportedSaid
“We have not one, but many, serious, unsolved problems at the core of generative AI — ranging from their tendency to confabulate (hallucinate) false information, to their inability to reliably interface with external tools like Wolfram Alpha, to the instability from month to month (which makes them poor candidates for engineering use in larger systems).”
What happened: OpenAI explicitly acknowledged continuing factual errors in its released model. The same section reports improvements, so persistence should not be confused with lack of progress. (OpenAI, )
Assessment: SupportedSaid
““Scaling” systems by making them larger has helped in some ways, but not others; we still really cannot guarantee that any given system will be honest, harmless, or helpful, rather than sycophantic, dishonest, toxic or biased.”
What happened: The study documents tasks where increasing scale worsened performance despite broader gains in prediction and other tasks. It also reports exceptions and trend reversals, so it does not establish that scaling generally fails. (Ian R. McKenzie and colleagues, arXiv, )
Assessment: SupportedSaid
“The technology we have now is built on autocompletion, not factuality.”
What happened: The report explicitly identifies predictive completion as GPT-4’s base objective. It separately reports factuality gains from post-training, supporting the foundation reading while qualifying a literal denial of factuality work. (OpenAI, arXiv, )
Assessment: SupportedSaid
“They will gladly replace artists, writers and any other role that they believe can be automated in a heartbeat.”
What happened: The report documents dismissal of the Spanish editorial staff and replacement with automated translations. It supplies a completed editorial substitution; no direct employer announcement was obtained. (Ars Technica, )
Assessment: SupportedSaid
“While AI will undoubtedly help us become more monetized and more interrupted by billion and trillion-dollar enterprises, it’s arguable that we haven’t already entered a digital society where the corporations are completely in control.”
What happened: The section confirms that ads directly within AI Overviews were available to US mobile users after testing, adding a placement inside the generated answer. That establishes AI-mediated commercialization and an additional commercial intrusion under the stated reading. (Google, )
Assessment: SupportedSaid
“AI might help profligate this kind of content, but you are completely delusional if you can’t see that the modern content mill of the internet is already full of crap specifically built to convince Google to give it more traffic.”
What happened: The investigation describes padded commercial copy, interviews a producer about writing for Google, and identifies mass SEO-oriented publishing at media outlets. This supports an established production practice; reputable reporting is used because no comprehensive contemporaneous primary measurement was found. (The Verge, )
Assessment: SupportedSaid
“We also know, for example, that Bing has defamed people, and it has misread articles as saying the opposite of what they actually say, in service of doing so.”
What happened: The same post identifies Turley's USA Today rebuttal among Bing's sources for the false allegation. It directly connects the reputational falsehood with the reversed source meaning. (Will Oremus, reporting on Twitter, )
Assessment: SupportedSaid
“Bing relies on GPT-4, and we (eg, in the scientific community) don’t have access to how GPT-4 works, and we don’t have access to what data its trained on (vital, since we know that systems can bias eg political thought and hiring decisions based on those undisclosed data) — that’s about as far away from transparency as we could be.”
What happened: The contemporaneous complaint identifies the specific missing disclosures. Its statement is corroborated by OpenAI's own report; the complaint itself is not an adjudicated regulatory finding. (Center for AI and Digital Policy, )
Assessment: SupportedSaid
“But their current, actual products are definitely not transparent, and at least in some ways, demonstrably not safe.”
What happened: The researchers describe Bing as a system they could test without access to its internals. They identify GPT-4 as its underlying model and distinguish available high-level descriptions from unavailable implementation details. (Kai Greshake and colleagues / arXiv, )
Assessment: SupportedSaid
“Meanwhile, ChatGPT plugins (produced by OpenAI, which they have a close tie with) open a wide range of security problems: those plugins can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all alarms to any security professional.”
What happened: The launch documentation establishes browsing and file-processing capabilities and reports fraudulent-email and prompt-injection findings without safeguards. It also describes sandboxing, network controls, and a GET-only browser, which limit the claim's scope. (OpenAI, )
Assessment: SupportedSaid
“Truthfulness is one of the major gaps in current systems, as evidenced by hallucinations, weird defamatory accusations etc.”
What happened: OpenAI explicitly identifies factual unreliability as a continuing limitation of its newly released leading model. The surrounding guidance calls for care and human review in consequential uses. (OpenAI, )