Claims checked against evidence by Gary Marcus, page 10 of 15
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Claims checked against evidence by Gary Marcus (1403)
Each of these claims was checked against dated evidence.
Page 10 of 15: claims 901 to 1000 of 1403, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: UnresolvedSaid
Observation
“I take the recital of facts in Musk’s lawsuit to be largely accurate. It is consistent with my own observations and with earlier reporting by The New York Times and The New Yorker.”
Our reading: The facts set out in Musk's lawsuit against OpenAI are largely accurate.
No horizon Evidence as of
Limitation: The original state complaint and later federal pleadings are different documents.
Source: Elon Musk sues OpenAI
Assessment: UnresolvedSaid
Observation
“Elon did not get what he paid and worked for.”
Our reading: Elon Musk did not get the OpenAI he paid for and worked for.
No horizon Evidence as of
Limitation: The record does not establish Musk's acceptance of the specific structure ultimately adopted [e3].
Source: Elon Musk sues OpenAI
Assessment: SupportedSaid
Observation
“That’s exactly what classical AI has been trying to do for decades, with only limited success. And what systems like Gemini still struggle with.”
Our reading: Classical AI has had only limited success in turning input into checkable logical form and reasoning over it, and systems like Gemini still struggle with it.
No horizon Evidence as of
Limitation: Limited success is a qualitative judgment about general coverage.
Source: No, RAG is probably not going to rescue the current situation
Assessment: UnresolvedSaid
Observation
“Investors put in over $100 billion into the driverless car industry, and so far have little to show far it.”
Our reading: Investors have put over $100 billion into driverless cars and so far have little to show for it.
No horizon Evidence as of
Limitation: Deployment is not revenue, profit, or investor return.
Source: No, RAG is probably not going to rescue the current situation
Assessment: SupportedSaid
Observation
“The trouble is that so far RAG has been hit and miss.”
Our reading: So far retrieval-augmented generation has produced inconsistent results.
No horizon Evidence as of
Limitation: Citation correctness is narrower than overall factual accuracy.
Source: No, RAG is probably not going to rescue the current situation
Assessment: UnresolvedSaid
Forecast
“RAG may reduce hallucinations somewhat, but it certainly won’t come close to eliminating them.”
Our reading: RAG may somewhat reduce hallucinations but will not come close to eliminating them.
No deadline stated Evidence as of
Limitation: The claim supplies no deadline or numerical definition of near-elimination.
Source: No, RAG is probably not going to rescue the current situation
Assessment: UnresolvedSaid
Observation
“There’s no reason to think that any such thing yet exists, nor that building such a comparison tool is any easier than solving AGI itself.”
Our reading: There is no reason to think that a technology yet exists for comparing an LLM's output with the text of a reliable source and determining with very high accuracy whether they say the same thing, nor that building such a comparison tool is any easier than solving AGI.
No horizon Evidence as of
Limitation: The compound claim combines an empirical existence question with an undefined difficulty comparison.
Source: No, RAG is probably not going to rescue the current situation
Assessment: SupportedSaid
Observation
“The road from research idea to reliable production remains steep.”
Our reading: Getting RAG from a research idea to reliable production remains difficult.
No horizon Evidence as of
Limitation: The implementation report covers three cases, not a representative industry survey.
Source: No, RAG is probably not going to rescue the current situation
Assessment: SupportedSaid
Observation
“Reality is that RAG is still in its infancy, a work in progress much as LLMs themselves are, an no more guaranteed to be correct. Like LLMs, RAG can appear spectacular at times, but there is no evidence that it is a cure-all.”
Our reading: RAG is immature, not guaranteed correct, and there is no evidence it is a cure-all for LLM problems.
No horizon Evidence as of
Limitation: The evidence does not give 'infancy' an objective maturity threshold or show that every RAG application was immature.
Source: No, RAG is probably not going to rescue the current situation
Assessment: UnresolvedSaid
Observation
“Gary Marcus has said from day 1 that LLMs are oversold.”
Our reading: Gary Marcus has said from day 1 that LLMs are oversold.
No horizon Evidence as of
Limitation: The documented criticism concerns GPT-2 and cannot establish his entire earlier record.
Source: Belief in magic may be declining
Assessment: MixedSaid
Observation
“So let me get this straight? Sora’s can’t reliably handle basic physics, ChatGPT had an unexplained meltdown, Gemini can’t even remember who was on Apollo 11, there are no formal guarantees that any of this will ever work, you yourself admit you don’t know how it works, or how much better it will get next time around, and you want another $7 trillion dollars?”
Our reading: ChatGPT had an unexplained meltdown.
No horizon Evidence as of
Limitation: The incident was initially unexplained. Reading the adjective as referring only to that initial stage would change the assessment.
Source: Belief in magic may be declining
Assessment: UnresolvedSaid
Observation
“The reality, though is that these systems have never been been stable.”
Our reading: Generative AI systems have never been stable.
No horizon Evidence as of
Limitation: Stable has an ordinary meaning, but its boundary is not specified.
Source: ChatGPT has gone berserk
Assessment: SupportedSaid
Observation
“Over the last few hours, people reporting having been report a variety of problems with ChatGPT:”
Our reading: Over the last few hours before the post, people reported a variety of problems with ChatGPT.
No horizon Evidence as of
Limitation: The evidence establishes reports, not the prevalence of malfunctions across users.
Source: ChatGPT has gone berserk
Assessment: SupportedSaid
Observation
“And one wants the system to have a nuanced understanding of what is even been asked. (That’s the clearly lacking now).”
Our reading: Current AI systems clearly lack a nuanced understanding of what is being asked.
No horizon Evidence as of
Limitation: The result depends on reading the claim in its immediate Gemini image-generation context.
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: UnresolvedSaid
Observation
“Current systems mix historical data, images from old films etc, all together and are way too unsophisticated to determine what would count as representative current sample.”
Our reading: Current AI systems mix historical and current data indiscriminately and are too unsophisticated to determine a representative current sample.
No horizon Evidence as of
Limitation: The demographic experiments predate the target period.
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: UnresolvedSaid
Observation
“But here’s the thing. The AI we have got now isn’t really up to the job. Getting the delicate balance here right is way outside current AI’s wheelhouse.”
Our reading: Current AI is not capable of getting the balance between cultural sensitivity and historical accuracy right.
No horizon Evidence as of
Limitation: The result depends on retaining the claim's general scope rather than narrowing it to Gemini.
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: UnresolvedSaid
Causal claim
“Getting AI to behave in a culturally-sensitive yet historically accurate way is hard for two reasons, one having to do with data and cultural change, the other with intelligence.”
Our reading: Making AI culturally sensitive yet historically accurate is hard because of skewed data/cultural change and AI's lack of intelligence.
No horizon Evidence as of
Limitation: Google's explanation is a company account rather than an independently reproduced causal investigation.
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: SupportedSaid
Observation
“It’s a general issue.”
Our reading: Gemini's historically inaccurate diversity in generated images is a general issue, not limited to popes.
No horizon Evidence as of
Limitation: The tests do not measure how frequently the problem occurred.
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: UnresolvedSaid
Observation
“It should try to clarify if it was unsure of the request. But current AI just ain’t that smart.”
Our reading: Current AI is not smart enough to behave as a sensible system should, for example by telling historical requests from contemporary ones and requests for real photos from requests for fictional characters, and by asking for clarification when unsure of a request.
No horizon Evidence as of
Limitation: Neither item supplies an evaluation matched to the February 2024 deployments and the full set of request distinctions [e6, e7].
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: SupportedSaid
Observation
“You might have thought that the overactive guardrail problem would have been solved by now, what with billions more invested and ever-bigger models, not to mention the cultural backlash from the right. You’d be wrong.”
Our reading: Despite billions more in investment and bigger models, the overactive guardrail problem in AI chatbots has not been solved as of February 2024.
No horizon Evidence as of
Limitation: These examples establish persistence, not the frequency of excessive refusals across assistants [e1, e2].
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: UnresolvedSaid
Forecast
“We are about to have a whole generation of children educated by fake videos that are completely plausible to naive audiences yet biologically (or physically or culturally, etc) incorrect.”
Our reading: Soon, a whole generation of children will be educated by AI-generated fake videos that look plausible to naive viewers but are factually incorrect.
No deadline stated Evidence as of
Limitation: The scope words ('whole generation', 'educated by') are hard to measure.
Source: Absolutely amazing ant video augurs intellectual mayhem
Assessment: SupportedSaid
Forecast
“In short order, with Sora or some opensource counterparts that will emerge, scammers will start to make photorealistic fake videos simply to sell ads.”
Our reading: Open-source counterparts to Sora will emerge.
Settled before its deadline Evidence as of
Limitation: The conclusion depends on interpreting counterparts functionally, without an unstated requirement for Sora-level performance.
Source: Absolutely amazing ant video augurs intellectual mayhem
Assessment: SupportedSaid
Observation
“… but, um, ants have six legs.”
Our reading: A Sora-generated ant video got the ants' anatomy wrong: it did not show the six legs real ants have.
No horizon Evidence as of
Limitation: The four-leg count comes from a secondary blog post. Nobody in this review watched the video and counted the legs.
Source: Absolutely amazing ant video augurs intellectual mayhem
Assessment: SupportedSaid
Observation
“In brief, an Air Canada chatbot confabulated (“hallucinated”) an answer – telling a customer that bereavement trips would be refunded.”
Our reading: An Air Canada chatbot hallucinated an answer telling a customer that bereavement trips would be refunded.
No horizon Evidence as of
Limitation: Support depends on reading refunded as including a partial refund and hallucinated as a description of incorrect policy information.
Assessment: UnresolvedSaid
Observation
“But it’s drawing the wrong generalization.”
Our reading: Sora arguably shows some generalization, but in the monkey-chess case it is drawing the wrong generalization.
No horizon Evidence as of
Limitation: The evidence packet has no independent description or analysis of the original monkey-chess video.
Source: Sora Can’t Handle the Truth
Assessment: SupportedSaid
Observation
“Gary Marcus is astonished that some people still haven’t recognized that generative AI has struggled mightily when it comes to world models, high level reasoning and factuality, even as evidence for that conclusion continues to pour in, even with data of ever-greater magnitude and sophistication..”
Our reading: Generative AI has struggled mightily with world models, high-level reasoning and factuality, even with ever more data.
No horizon Evidence as of
Limitation: The severity term is qualitative, so the outcome depends partly on the stated reading.
Source: Sora Can’t Handle the Truth
Assessment: UnresolvedSaid
Observation
“Sora fundamentally revolves around trying to take pixels and reconstruct what pixels might be nearby in space and time. The forest is never perceived apart from the trees, and the chessboard is never perceived apart from its constituent squares.”
Our reading: Sora fundamentally works by reconstructing nearby pixels in space and time rather than perceiving whole objects.
No horizon Evidence as of
Limitation: The cited review appeared after the post and is used only as context about the original model.
Source: Sora Can’t Handle the Truth
Assessment: SupportedSaid
Conditional forecast
“We will, I am certain, see more systemic glitches as more people have access.”
Our reading: As more people gain access to Sora, more systemic glitches will be found.
Settled before its deadline Evidence as of
Only if: as more people get access
Source: Sora’s Surreal Physics
Assessment: UnresolvedSaid
Forecast
“More data won’t solve that problem.”
Our reading: More training data will not solve Sora's hallucination-like physics glitches.
No deadline stated Evidence as of
Limitation: The research experiments test models other than OpenAI's Sora and do not establish what all larger datasets could achieve [e1, e2].
Source: Sora’s Surreal Physics
Assessment: UnresolvedSaid
Forecast
“And importantly, I predict that many will be hard to remedy.”
Our reading: Many of Sora's physics glitches will be hard to fix.
No deadline stated Evidence as of
Limitation: Public development disclosures provide little detail about targeted repair effort.
Source: Sora’s Surreal Physics
Assessment: UnresolvedSaid
Observation
“If the system really is being “programmed in English” it is not always doing what is being asked to do:”
Our reading: If ChatGPT/GPT-4 really is being "programmed in English" through its allegedly leaked system prompt, it is not always doing what it is asked to do.
No horizon Evidence as of
Limitation: An absent authentication record does not show that the alleged prompt was fabricated.
Source: The hottest new programming language is English! Or maybe not.
Assessment: SupportedSaid
Observation
“But the truth is it doesn’t work that well yet.”
Our reading: Programming in English with LLMs does not yet work very well.
No horizon Evidence as of
Limitation: The competitive-programming evaluation does not establish a representative success rate for everyday programming or direct control through English prompts [e1].
Source: The hottest new programming language is English! Or maybe not.
Assessment: UnresolvedSaid
Observation
“Much of what Reid Southen, Katie Conrad, and I recently described is in flagrant violation of those directives.”
Our reading: Much of what Marcus, Reid Southen and Katie Conrad recently described is in flagrant violation of the directives in ChatGPT's allegedly leaked system prompt.
No horizon Evidence as of
Limitation: This assessment concerns compliance with instructions, not whether an image legally infringes copyright.
Source: The hottest new programming language is English! Or maybe not.
Assessment: UnresolvedSaid
Observation
“One winner out of three to be exact.”
Our reading: After several retries, Gemini produced only one correct horse-riding-astronaut image out of three.
No horizon Evidence as of
Limitation: This concerns one historical response, not Gemini's general success rate.
Source: Horse rides Astronaut, redux
Assessment: UnresolvedSaid
Causal claim
“In other words, the problem was with its language understanding, rather than with illustration per se. The model was perfectly capable of drawing a horse riding an astronaut, it just didn’t know that the phrase horse rides astronaut was a request for a horse riding an astronaut.”
Our reading: DALL-E's failure on 'horse rides astronaut' was due to deficient language understanding, not illustration ability.
No horizon Evidence as of
Limitation: The original images and complete sampling records were not available in the retrieved text.
Source: Horse rides Astronaut, redux
Assessment: UnresolvedSaid
Observation
“I think we can safely reach the same conclusion as before. Gemini can draw the desired image. But it’s still like pulling teeth to get there.”
Our reading: Gemini can draw a horse riding an astronaut, but getting it to do so is very difficult.
No horizon Evidence as of
Limitation: A verified difficult session would support the local observation more strongly than a general claim about typical performance.
Source: Horse rides Astronaut, redux
Assessment: ContradictedSaid
Observation
“In some ways we have made progress, but in the ones I keep harping on—factuality and compositionality—we have not.”
Our reading: Generative AI has made no progress on factuality and compositionality since 2022, despite progress in other respects.
No horizon Evidence as of
Limitation: The verdict depends on reading progress as measurable improvement rather than complete resolution.
Source: Horse rides Astronaut, redux
Assessment: UnresolvedSaid
Observation
“In our own brief experiments with GPT-3 (OpenAI has refused us proper scientific access for over a year) we found cases like the following, which reflects a complete failure to understand human biology.”
Our reading: In Marcus and Davis's experiments, GPT-3 produced outputs showing a complete failure to understand human biology.
No horizon Evidence as of
Limitation: The local-error reading is narrower than a claim that GPT-3 understood nothing about biology.
Source: Statistics versus Understanding: The Essence of What Ails Generative AI
Assessment: UnresolvedSaid
Causal claim
“That’s why statistically improbable requests like horses riding astronauts have always been challenging for generative image systems.”
Our reading: Because they rely on statistics instead of understanding, generative image systems have always struggled with statistically improbable requests such as horses riding astronauts.
No horizon Evidence as of
Limitation: The most directly relevant study concerns one model.
Source: Statistics versus Understanding: The Essence of What Ails Generative AI
Assessment: UnresolvedSaid
Causal claim
“Once again statistical frequency (most guitarists play righthanded) won out over linguistic understanding, to the consternation of Hendrix fans everywhere.”
Our reading: In Wyatt Walls's left-handed guitarist example, statistical frequency (most guitarists play right-handed) once again won out over linguistic understanding.
No horizon Evidence as of
Limitation: Image-based tweets could not be inspected.
Source: Statistics versus Understanding: The Essence of What Ails Generative AI
Assessment: SupportedSaid
Observation
“And, let’s be honest folks, the “alignment problem” isn’t remotely solved.”
Our reading: The AI alignment problem is not remotely solved.
No horizon Evidence as of
Limitation: The decisive study was a preprint using deliberately constructed backdoors.
Source: Seven reasons why the world should say No to Sam Altman
Assessment: UnresolvedSaid
Observation
“Negative externalities, essentially none of which OpenAI is absorbing, like the cost of dis- and misinformation, cybercrimes, and the latest, fake books, a scam that has recently hurt at least three authors I know over the last two months.”
Our reading: OpenAI absorbs essentially none of the negative externalities of its products, such as misinformation, cybercrime, and fake books.
No horizon Evidence as of
Limitation: Private payments and remediation costs may not be publicly disclosed.
Source: Seven reasons why the world should say No to Sam Altman
Assessment: UnresolvedSaid
Conditional forecast
“They are already a target for attack, and as long as they are successful they will continue to be; if they have direct access to user devices, the risks of them getting hacked will increase, and the spoils of war for the attackers will increase, since it enables essentially infinite mischief (e.g., draining bank balances, stealing credit card numbers, placing fraudulent transactions, extortion based on private photos, holding devices for ransom, and so on).”
Our reading: If OpenAI gains direct access to user devices, the risk of it being hacked and the rewards to attackers will increase.
No deadline stated Evidence as of
Only if: if OpenAI has direct access to user devices
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: UnresolvedSaid
Causal claim
“Chatbots are struggling to make the money people envision, so he is doubling down to (a) agents, and (b) a multitrillion-dollar chips-and-energy infrastructure play.”
Our reading: Chatbots are struggling to make the money people envision, so Altman is doubling down on agents and on a multitrillion-dollar chips-and-energy infrastructure play.
No horizon Evidence as of
Limitation: The financial measure and expected result are unspecified.
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: SupportedSaid
Causal claim
“In a system that can write emails, make appointments, sign checks, etc, the consequences of unreliability, discomprehension, and hallucination all escalate.”
Our reading: When LLMs can take actions like writing emails or signing checks, the consequences of their unreliability and hallucinations escalate.
No horizon Evidence as of
Limitation: The evidence supports increased potential consequences, not inevitable harm from every action.
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: SupportedSaid
Forecast
“This list is probably not exhaustive and will continue to grow.”
Our reading: The list of OpenAI/LLM security issues will continue to grow.
Settled before its deadline Evidence as of
Limitation: This establishes additional vulnerability disclosure, not a rate of future incidents.
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: SupportedSaid
Observation
“More evidence against his claims just keeps piling up.”
Our reading: Evidence against Hinton's claims that GenAI understands and doesn't store things keeps accumulating.
No horizon Evidence as of
Limitation: The result depends on reading the statement as an evidentiary observation rather than a conclusive theory of understanding.
Source: Further Trouble in Hinton City
Assessment: UnresolvedSaid
Observation
“GenAI systems do effectively store things. (In this case, canonical video game controller images.)”
Our reading: GenAI systems effectively store things such as canonical video game controller images.
No horizon Evidence as of
Limitation: If the controller parenthetical is treated only as an illustration, the general storage proposition is supported.
Source: Further Trouble in Hinton City
Assessment: UnresolvedSaid
Causal claim
“Noncanonical situations continue to remain difficult for generative AI systems, precisely because frequent statistics are a mediocre substitute for deep understanding.”
Our reading: Generative AI systems still struggle with noncanonical situations because frequency statistics are a poor substitute for deep understanding.
No horizon Evidence as of
Limitation: Both studies predate the target period and cover narrower image-generation settings.
Source: Further Trouble in Hinton City
Assessment: UnresolvedSaid
Observation
“They (often) don’t come up with a novel independent design that would meet the requirements.”
Our reading: GenAI systems often don't come up with a novel independent design that would meet the requirements.
No horizon Evidence as of
Limitation: The packet lacks representative evaluations jointly measuring novelty and functional compliance.
Source: Further Trouble in Hinton City
Assessment: SupportedSaid
Observation
“Reliability remain a problem.”
Our reading: Reliability remains a problem for large language models.
No horizon Evidence as of
Limitation: Benchmark performance varies by task and scoring method.
Assessment: SupportedSaid
Observation
“Scale isn’t everything, and hasn’t and won’t be enough to solve any of the problems I just listed above.”
Our reading: Scaling has not been enough to solve the problems of understanding, compositionality, negation, and reliability in LLMs.
No horizon Evidence as of
Limitation: The result depends on treating understanding as observable interpretation and generalization.
Assessment: UnresolvedSaid
Forecast
“Scale isn’t everything, and hasn’t and won’t be enough to solve any of the problems I just listed above.”
Our reading: Scaling will not be enough to solve the problems of understanding, compositionality, negation, and reliability in LLMs.
No deadline stated Evidence as of
Limitation: The boundary between scaling and a new method is unclear.
Assessment: SupportedSaid
Observation
“Compositionality remains a problem. So does negation.”
Our reading: Compositionality and negation remain problems for large language models.
No horizon Evidence as of
Limitation: Results depend on task, model version and prompt.
Assessment: UnresolvedSaid
Forecast
“But anyone who fears that GPT-5 is going to convey an immense advantage in military strategy has got to be kidding.”
Our reading: GPT-5 will not convey an immense advantage in military strategy.
No deadline stated Evidence as of
Limitation: Public evidence may omit classified deployments.
Assessment: MixedSaid
Observation
“Special-purpose AI has been great for static, easily-simulated games with rules that have been stable for centuries, like Chess, Go, and even Diplomacy, but never been great at dynamic environments in which the environment itself is unpredictable.”
Our reading: Special-purpose AI has been great for static, easily simulated games with rules that have been stable for centuries, such as chess, Go and even Diplomacy, but has never been great at dynamic environments in which the environment itself is unpredictable.
No horizon Evidence as of
Limitation: The distinction between environmental uncertainty and unrestricted task change affects the result.
Assessment: UnresolvedSaid
Observation
“Even with a lot of data, GPT-4 is playing chess at around the level where (top) chess programs were in 1976, not even close to current state of the art.”
Our reading: GPT-4 plays chess at about the level of top chess programs in 1976, far from the current state of the art.
No horizon Evidence as of
Limitation: The 1976 tournament figure comes from a secondary account [e2].
Assessment: SupportedSaid
Observation
“It’s notorious for making illegal moves in chess.”
Our reading: ChatGPT is notorious for making illegal moves in chess.
No horizon Evidence as of
Limitation: Notoriety is qualitative.
Assessment: MixedSaid
Observation
“The New York Times versus OpenAI lawsuit made that abundantly clear, with a hundred examples like this one, in which ChatGPT reconstructed entire stretches of text (red) from brief prompts2 (black text in top line):”
Our reading: The NYT v. OpenAI lawsuit showed about a hundred examples of ChatGPT reconstructing entire stretches of text from brief prompts.
No horizon Evidence as of
Limitation: Both filings are party submissions, not judicial findings about experimental validity.
Assessment: UnresolvedSaid
Causal claim
““Given vast datasets, LLMs approximate well, but their understanding is at best superficial. That’s *why* they are unreliable, and unstable, hallucinate, are constitutionally unable to fact check, etc.””
Our reading: Given vast datasets, LLMs approximate well, but their understanding is at best superficial, and that is why they are unreliable, unstable, hallucinate, are constitutionally unable to fact-check, and so on.
No horizon Evidence as of
Limitation: Partial error detection is not reliable verification of arbitrary claims.
Assessment: SupportedSaid
Observation
“But the variety of the examples along with the technical literature on memorization and data leakage in large language models make clear that this kind of reconstruction certainly can happen with some regularity.”
Our reading: Reconstruction of training text by large language models can happen with some regularity.
No horizon Evidence as of
Limitation: Marcus supplies no numerical meaning for 'some regularity'.
Assessment: SupportedSaid
Observation
“LLMs don’t store things in simple databases, but when presented with enough data, their regeneration of distributed bits has demonstrably duplicated lengthy training materials multiple times, a phenomenon that is called memorization in the field.”
Our reading: Given enough data, LLMs have demonstrably duplicated lengthy training materials multiple times (memorization).
No horizon Evidence as of
Limitation: Extraction methods can involve deliberate prompting and specialized decoding.
Assessment: UnresolvedSaid
Observation
“OpenAI will no doubt by now have filtered this specific example so that it can no longer be replicated, now that it has appeared in a lawsuit.”
Our reading: OpenAI has no doubt by now filtered this specific NYT-lawsuit example so that it can no longer be replicated, now that it has appeared in a lawsuit.
No horizon Evidence as of
Limitation: The relevant generation setup is unspecified.
Assessment: UnresolvedSaid
Causal claim
“Because of the immense amounts of data they are now trained on, and because of the immense number of parameters on which they are trained, that reconstruction can in fact sometimes come very close to memorization.”
Our reading: Because of their huge training data and parameter counts, LLM reconstruction of inputs sometimes comes very close to memorization.
No horizon Evidence as of
Limitation: This outcome concerns the full causal sentence, not whether LLMs memorize.
Assessment: UnresolvedSaid
Observation
“LLMs do (sometimes) regurgitate large chunks of effectively stored text, and they don’t understand anything at a deep level.”
Our reading: LLMs sometimes regurgitate large chunks of effectively stored text and do not understand anything at a deep level.
No horizon Evidence as of
Limitation: Closed-model training membership is inferred from matching pre-existing material rather than inspection of the complete private training corpus.
Assessment: SupportedSaid
Observation
“Neural networks do effectively memorize some of what they are trained on, and indeed there is a whole subfield now about the topic.”
Our reading: Neural networks effectively memorize some of what they are trained on, and there is now a whole subfield devoted to this topic.
No horizon Evidence as of
Limitation: For closed models, the researchers used an auxiliary corpus because the complete training data were unavailable.
Assessment: SupportedSaid
Observation
“An entire cast of deepfaked people include a deep faked CFO scammed a company out $25 million.”
Our reading: Deepfaked participants, including a deepfaked CFO, on a video call scammed a company out of $25 million.
No horizon Evidence as of
Limitation: Support depends on reading $25 million as approximate, not exact.
Source: Deepfaked Shit is Getting Real
Assessment: SupportedSaid
Observation
“Even scarier is that the employee who was tricked was suspicious, not naive, and still fell for it:”
Our reading: The deepfake scam worked even on an employee who was suspicious rather than naive.
No horizon Evidence as of
Limitation: The account of suspicion comes through police reporting, rather than a direct employee interview.
Source: Deepfaked Shit is Getting Real
Assessment: SupportedSaid
Observation
“I am here to tell you that OpenAI’s research, analyzed carefully, shows no such thing, and that there is plenty of reason to be concerned.”
Our reading: OpenAI's bioweapons research, analyzed carefully, does not show that GPT-4 poses at most a slight risk of helping create biological threats, and there is plenty of reason to be concerned.
No horizon Evidence as of
Limitation: OpenAI itself acknowledged uncertainty; Marcus's criticism should not imply that the study concealed all these limitations.
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: SupportedSaid
Observation
“For one thing, in the expert population, the effect is in the predicted direction for all five tasks (ideation, acquisition, magnification, formulation, and release).”
Our reading: In OpenAI's threshold analysis, among experts the effect was in the predicted direction for all five tasks (ideation, acquisition, magnification, formulation and release).
No horizon Evidence as of
Limitation: These observed threshold proportions alone do not establish statistical significance or real-world biological threat capability.
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: SupportedSaid
Observation
“In fact, if you simply collapsed all the measurements of accuracy, and did the single most obvious test here, a simple t-test, the results would (as Footnote C implies) be significant.”
Our reading: If all the accuracy measurements in OpenAI's study were collapsed and tested with a simple t-test, the results would be statistically significant, as Footnote C implies.
No horizon Evidence as of
Limitation: The result does not establish that this analysis was uniquely appropriate or that multiple-comparison adjustment was unnecessary.
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: SupportedSaid
Observation
“Any fool can simply eyeball these data and see that in virtually every condition, people who have access both to the internet and LLMs (darker bars) are more accurate than people without (lighter bars).”
Our reading: In OpenAI's study, participants with both internet and LLM access were more accurate than those without in virtually every condition.
No horizon Evidence as of
Limitation: Higher observed means do not establish significance or practical danger.
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: UnresolvedSaid
Observation
“At present, we do not know.”
Our reading: It is currently unknown whether open-source models without guardrails are more useful to bad actors than commercial models with guardrails.
No horizon Evidence as of
Limitation: Earlier research is context because the claim concerns the knowledge state on February 4.
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: UnresolvedSaid
Observation
“It is shocking that neither OpenAI nor the media noted this (they also did not discuss in any detail one other paper that reached a somewhat similar conclusion, by Kevin Esvelt’s lab).”
Our reading: Neither OpenAI nor the media noted the fourfold difference in expert success on pathogen formulation, nor discussed in detail a similar paper from Kevin Esvelt's lab.
No horizon Evidence as of
Limitation: The supplied evidence items omit OpenAI's publication and do not provide a sufficiently comprehensive contemporary media record.
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: ContradictedSaid
Observation
“As a journal reviewer, I would also note that they reported neither the p value they obtained, nor the test of statistical significance they applied; in a competent peer-reviewed journal, that’s a no-no.”
Our reading: OpenAI's paper reported neither the p value obtained nor the significance test applied, which would be unacceptable in a competent peer-reviewed journal.
No horizon Evidence as of
Limitation: The p-values are in a linked CSV rather than stated directly in footnote C.
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: UnresolvedSaid
Observation
“Not every image looks like this, but it’s pretty clear after a few minutes experimentation from a friend that (a) Bard makes fantastic images but that (b) it is drawing from copyrighted sources and (c) as capable as the other systems Reid Southen and I looked at in returning derivative artwork that appears to infringe without attribution.”
Our reading: Based on a friend's few minutes of experimentation, it is pretty clear that Bard makes fantastic images, draws from copyrighted sources, and is as capable as the other systems Marcus and Reid Southen looked at in returning derivative artwork that appears to infringe without attribution, though not every image looks like this.
No horizon Evidence as of
Limitation: Plain-text sources do not provide a complete authenticated visual record.
Assessment: SupportedSaid
Forecast
“Gary Marcus is standing firmly by his prediction that 2024 will be the year of Generative AI litigation.”
Our reading: 2024 will be the year of generative AI litigation.
Deadline passed Evidence as of
Limitation: “The year of” supplies no numerical threshold; support follows the contract's ordinary reading of substantial, sustained litigation.
Assessment: SupportedSaid
Observation
“What unifies all of the above is that current systems are good at local coherence, between words, and between pixels, but not at lining up their outputs with a global comprehension of the world.”
Our reading: Current systems are good at local coherence, both between words and between pixels, but are not good at lining up their outputs with a global comprehension of the world.
No horizon Evidence as of
Limitation: Support is qualitative and depends on the stated behavioral interpretation.
Source: One GenAI image; Ten Errors
Assessment: UnresolvedSaid
Observation
“Third, at least so far OpenAI lacks both profits and a moat.”
Our reading: As of January 2024, OpenAI has neither profits nor a competitive moat.
No horizon Evidence as of
Limitation: Earlier losses cannot be carried forward as proof of current losses.
Assessment: UnresolvedSaid
Forecast
“Such errors clearly aren’t disappearing any time soon.”
Our reading: Errors of discomprehension in neural-network-based systems will not disappear any time soon.
No deadline stated Evidence as of
Limitation: Unresolved does not mean the later evidence runs against the forecast.
Assessment: UnresolvedSaid
Observation
“Examples like these, in which ChatGPT confidently generates an image that does not actually meet requirements, are nicely complement the visual hallucinations that Ernest Davis and I reported here in October, in which visual ChatGPT misinterpreted various images:”
Our reading: In examples like those shown, ChatGPT confidently generated images that did not actually meet requirements, and in examples reported in October, visual ChatGPT misinterpreted various images.
No horizon Evidence as of
Limitation: This is an access and verification gap, not evidence that Marcus's descriptions were false.
Assessment: SupportedSaid
Forecast
“We can of course expect related problems of discomprehension in efforts to use these systems to address copyright issues, as I wrote earlier this week.”
Our reading: Efforts to use generative AI systems to address copyright issues will run into problems of discomprehension.
Settled before its deadline Evidence as of
Limitation: The decisive study is a preprint.
Assessment: UnresolvedSaid
Observation
“It might be trivial to do it for the most popular characters, presented in the most canonical ways, but getting it right for all copyright materials is an entirely different matter.”
Our reading: Building a classifier that detects copyrighted material might be trivial for the most popular characters presented in the most canonical ways, but getting it right for all copyrighted materials is an entirely different matter.
No horizon Evidence as of
Limitation: The packet contains no qualifying evaluation of the January 2024 comparison.
Source: No, multimodal ChatGPT is not going to “trivially” solve Generative AI's copyright problems
Assessment: UnresolvedSaid
Observation
“Reid spent about 45 minutes looking into this earlier today (why not more? see below) with images generated by Midjourney, and it quickly became apparent that using ChatGPT as intern to trace down sources is, well, pretty intermittent.”
Our reading: In a 45-minute test, multimodal ChatGPT was unreliable at identifying the copyrighted sources of Midjourney-generated images.
No horizon Evidence as of
Limitation: The packet has no independent record of the images Southen tested or of ChatGPT's answers to them.
Source: No, multimodal ChatGPT is not going to “trivially” solve Generative AI's copyright problems
Assessment: SupportedSaid
Forecast
“More than that, GenAI developers won’t get off so easy, and lawyers looking at copyright infringement will still have plenty of work to do.”
Our reading: GenAI developers will not easily escape copyright problems, and copyright infringement lawyers will have plenty of work.
Settled before its deadline Evidence as of
Limitation: The assessment treats substantial contested proceedings as the ordinary meaning of plenty of legal work.
Source: No, multimodal ChatGPT is not going to “trivially” solve Generative AI's copyright problems
Assessment: UnresolvedSaid
Observation
“Notice, too, that Katie asked Bing “are they original?”, and Bing, untruthfully, but with a cozy emoticon, answered in the affirmative.”
Our reading: Bing falsely said its apparently infringing images were original.
No horizon Evidence as of
Limitation: The packet has no independent record of Bing's reply or of the generated images.
Source: DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
Assessment: UnresolvedSaid
Observation
“Fundamentally, virtually any guardrail has to thread a needle between the Scylla of being too restrictive and Charybdis of being too permissive. None thus far have done this effectively.”
Our reading: No AI guardrail so far has effectively balanced being neither too restrictive nor too permissive.
No horizon Evidence as of
Limitation: The cited evaluations cover selected models, prompts, and attacks.
Source: DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
Assessment: UnresolvedSaid
Causal claim
“Any one example can be filtered out, but there are too many examples, and current systems are too erratic to reliably get things right.”
Our reading: Because there are too many examples and current systems are too erratic, guardrails cannot reliably get things right even though any single example can be filtered.
No horizon Evidence as of
Limitation: The controlled defense study is contextual because its experiments predate the target period.
Source: DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
Assessment: UnresolvedSaid
Observation
“Guardrail builders generally wind up playing whack-a-mole.”
Our reading: Guardrail builders generally end up patching individual failures one by one instead of solving the problem in general.
No horizon Evidence as of
Limitation: Public summaries provide less detail than internal remediation records.
Source: DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
Assessment: UnresolvedSaid
Observation
“Other prompts still elicited apparent copyright infringements, where new filters did not suffice.”
Our reading: Even after the new filters, some prompts to Bing's DALL-E 3 still produced apparent copyright infringements.
No horizon Evidence as of
Limitation: The independent demonstration lacks authenticated generation timestamps and filter-version information.
Source: DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
Assessment: UnresolvedSaid
Forecast
“Since that would violate the Berne Convention and infuriate every “content provider” (from individual artists and writers to major studios and publishers) ever, we hope and expect that this exemption will not be granted.”
Our reading: The UK will not grant OpenAI the exemption from copyright responsibility it requested.
No deadline stated Evidence as of
Limitation: The exemption Marcus described is not precisely identified.
Source: DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
Assessment: SupportedSaid
Observation
“The opposite problem happens too; guardrails can wind up too permissive. This is still happening over a year into the chatbot revolution.”
Our reading: Guardrails sometimes end up too permissive, and this is still happening over a year into the chatbot revolution.
No horizon Evidence as of
Limitation: Support rests on a primary researcher summary rather than fully reported contemporary replication results.
Source: DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
Assessment: UnresolvedSaid
Observation
“It’s likewise a flaw that that current AI cannot learn without a tendency to inadvertently regurgitate big bits of text and images that look uncannily like their sources, but not a logical necessity.”
Our reading: Current AI cannot learn without a tendency to inadvertently regurgitate big bits of text and images that look uncannily like their sources, which is a flaw but not a logical necessity.
No horizon Evidence as of
Limitation: The extraction studies do not establish an unavoidable tendency across contemporary text and image generators [e1, e2].
Assessment: UnresolvedSaid
Forecast
“LLMs may eventually be replaced by better technologies, but for now, they aren’t going anywhere, even if some of the commercial purveyors go out of business.”
Our reading: LLMs will not disappear for now even if some commercial providers go out of business, though they may eventually be replaced by better technologies.
No deadline stated Evidence as of
Limitation: Unresolved does not mean the observed trend contradicts the forecast.
Assessment: UnresolvedSaid
Observation
“OpenAI’s lobbying campaign, simply put, is based on a false dichotomy (give everything to us free or we will die)—and also a threat: either we get to use all the existing IP we want for free, or you won’t get to generative AI anymore.”
Our reading: OpenAI's copyright lobbying campaign rests on a false dichotomy and an implied threat that generative AI will go away without free use of IP.
No horizon Evidence as of
Limitation: This is an evidence-access limitation, not a finding that Marcus’s characterization is accurate.
Assessment: SupportedSaid
Observation
“To begin with, the threat is empty: open source LLMs are already out in the wild, and a large number of people know how to build them.”
Our reading: OpenAI's threat that generative AI would disappear is empty because open source LLMs already exist and many people can build them.
No horizon Evidence as of
Limitation: The result uses the ordinary public-model reading of open source.
Assessment: UnresolvedSaid
Observation
“And worse, infringement can happen even the user isn’t looking to infringe and doesn’t mention any character or film by name:”
Our reading: DALL-E can produce infringing images even when the user does not intend to infringe or name any character or film.
No horizon Evidence as of
Limitation: Reading infringement merely as recognizable copyrighted imagery would weaken the claim's stated legal commitment.
Source: Things are about to get a lot worse for Generative AI
Assessment: SupportedSaid
Observation
“Generative AI systems like DALL-E and ChatGPT have been trained on copyrighted materials;”
Our reading: Generative AI systems such as DALL-E and ChatGPT were trained on copyrighted materials.
No horizon Evidence as of
Limitation: The records do not provide a complete inventory of training works or establish which material entered every model version.
Source: Things are about to get a lot worse for Generative AI
Assessment: UnresolvedSaid
Observation
“OpenAI’s image software (which we accessed through Bing) is perfectly capable of verbatim and near-verbatim repetition of sources as well.”
Our reading: OpenAI's image generator can reproduce source images verbatim or near-verbatim.
No horizon Evidence as of
Limitation: A broader reading that counts recognizable character designs as near-verbatim sources would require different criteria.
Source: Things are about to get a lot worse for Generative AI
Assessment: UnresolvedSaid
Observation
“The output was not marked as being directly premised on copyrighted work”
Our reading: Midjourney's output was not labelled as being directly based on copyrighted work.
No horizon Evidence as of
Limitation: Absence of labelling is hard to document from secondary sources.
Source: An artist fights back, and Midjourney has embarrassed themselves