Claims that can’t be checked by Gary Marcus, page 17 of 24
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Claims that can’t be checked by Gary Marcus (2353)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 17 of 24: claims 1601 to 1700 of 2353, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Forecast
“At the same time, the system seemed yesterday (I imagine it will be partly patched soon) wildly too restrictive on some other queries:”
Our reading: Gemini's overly restrictive responses to some queries will likely be partly patched soon.
No horizon
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In fact, I would go as far as to say that I actually think that it is good that Google is trying here (albeit clearly ineptly).”
Our reading: It is good that Google is trying to counter minority underrepresentation in generated images, though it is doing so ineptly.
No horizon
Qualifier: albeit ineptly
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Increased transparency is essential.”
Our reading: Increased transparency about LLM training data and hidden system prompts is essential.
No horizon
Qualifier: about training data and hidden prompts
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: Can’t be checkedSaid
Rhetoric
“We are in for a wild ride.”
Our reading: Society faces turbulent times from AI-generated video.
No horizon
Source: Absolutely amazing ant video augurs intellectual mayhem
Assessment: Can’t be checkedSaid
Forecast
“A major educational resource will likely be spoiled practically overnight.”
Our reading: YouTube is likely to be ruined as a major educational resource almost overnight by AI-generated fake videos.
No horizon
Qualifier: due to AI-generated fake videos
Source: Absolutely amazing ant video augurs intellectual mayhem
Assessment: Can’t be checkedSaid
Conditional forecast
“If other judges rule the same way, and hold companies responsible for the nonsense their chatbots say, one of the biggest alleged use cases for large language models could dry up, fast.”
Our reading: If other judges rule the same way and hold companies responsible for the nonsense their chatbots say, one of the biggest alleged use cases for large language models could dry up fast.
No horizon
Only if: other judges rule the same way
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In the tech industry, this should be the top story.”
Our reading: The Air Canada chatbot ruling should be the top story in the tech industry.
No horizon
Qualifier: in the tech industry
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Nor should he have.”
Our reading: The tribunal judge was right not to accept Air Canada's argument that its chatbot was a separate legal entity responsible for its own actions.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“It’s probably not fair to blame the weird board on a lack of data.”
Our reading: Sora's erroneous chessboard is probably not due to a lack of training data.
No horizon
Source: Sora Can’t Handle the Truth
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“From AGI perspective, however, this is an immense warning sign.”
Our reading: From an AGI perspective, Sora's fixation on the local at the expense of a deep representation of the global is an immense warning sign.
No horizon
Scope: from an AGI perspective
Source: Sora Can’t Handle the Truth
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The sooner the AI industry -- and the world at large — recognizes this, the better.”
Our reading: The AI industry and the world should recognize soon that generative AI shares a persistent factuality and comprehension problem.
No horizon
Source: Sora Can’t Handle the Truth
Assessment: Can’t be checkedSaid
Observation
“Artists are presumably getting really screwed here.”
Our reading: Artists are probably being badly harmed by the way Sora was trained.
No horizon
Source: Sora’s Surreal Physics
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“As a solution to artificial general intelligence, though, I see it as a distraction.”
Our reading: In Marcus's view, Sora is a distraction as a solution to artificial general intelligence.
No horizon
Qualifier: I see it as
Source: Sora’s Surreal Physics
Assessment: Can’t be checkedSaid
Observation
“Sora is fantastic, but it is akin to morphing and splicing, rather than a path to the physical reasoning we would need for AGI. It is a model of how images change over time, not a model of what entities do in the world.”
Our reading: Sora is akin to morphing and splicing of images, not a path to the physical reasoning needed for AGI.
No horizon
Source: Sora’s Surreal Physics
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If a system can’t at all handle with the permanence of objects, I am not sure we should even call it a world model at all.”
Our reading: If a system can't handle the permanence of objects at all, Marcus is not sure it should even be called a world model.
No horizon
Only if: if a system can't at all handle the permanence of objects
Source: Sora’s Surreal Physics
Assessment: Can’t be checkedSaid
Observation
“I would not at all be surprised if there also had been lots of training on YouTube visited, and various copyrighted materials.”
Our reading: Sora may well have been trained on lots of YouTube video and various copyrighted materials.
No horizon
Source: Sora’s Surreal Physics
Assessment: Can’t be checkedSaid
Observation
“In my view, these claims — about AGI and world models — are hyperbolic, and unlikely to be true.”
Our reading: Claims that Sora amounts to AGI, vindicates scaling laws, or constitutes a world model are hyperbolic and unlikely to be true.
No horizon
Source: Sora’s Surreal Physics
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“As a technology for video artists that’s fine, if they choose to use it; the occasional surrealism may even be an advantage for some purposes (like music videos).”
Our reading: As a technology for video artists Sora is fine if they choose to use it, and its occasional surrealism may even be an advantage for some purposes such as music videos.
No horizon
Qualifier: if they choose to use it
Source: Sora’s Surreal Physics
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In the new regime, caveat emptor.”
Our reading: People programming via LLMs in English should beware of the risks.
No horizon
Source: The hottest new programming language is English! Or maybe not.
Assessment: Can’t be checkedSaid
Observation
“DALL-E 2 is, I dare say, not as smart as it seems.”
Our reading: DALL-E 2 is not as smart as it appears.
No horizon
Source: Horse rides Astronaut, redux
Assessment: Can’t be checkedSaid
Observation
“The problem with “Foundation Models” (a common term for Generative AI) is that they have never provided the firm, reliable foundation that their name implies.”
Our reading: Foundation models (generative AI) have never provided the firm, reliable foundation their name implies.
No horizon
Source: Statistics versus Understanding: The Essence of What Ails Generative AI
Assessment: Can’t be checkedSaid
Observation
“Generative AI systems have always tried to use statistics as a proxy for deeper understanding, and it’s never entirely worked.”
Our reading: Generative AI systems have always tried to use statistics as a proxy for deeper understanding, and it has never entirely worked.
No horizon
Qualifier: always
Source: Statistics versus Understanding: The Essence of What Ails Generative AI
Assessment: Can’t be checkedSaid
Observation
“GPT-3 has no idea what grape juice is, or what cranberry juice is, or what pouring, sniffing, smelling, or drinking are, or what it means to be dead.”
Our reading: GPT-3 has no idea what grape juice or cranberry juice is, what pouring, sniffing, smelling or drinking are, or what it means to be dead.
No horizon
Scope: GPT-3, in the grape-juice example
Source: Statistics versus Understanding: The Essence of What Ails Generative AI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Taking a real chance at wrecking both society and the environment on little more than vague speculation makes no sense.”
Our reading: Taking a real chance at wrecking both society and the environment on little more than vague speculation makes no sense.
No horizon
Qualifier: on little more than vague speculation
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Conditional forecast
“If the battles over resources became acute enough, actual war could ensue.”
Our reading: If competition over resources for AI chips became acute enough, actual war could follow.
No horizon
Only if: if battles over resources for AI chips became acute enough
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Conditional forecast
“Even if AI doesn’t kill all or many of us directly, as some people fear, the collapse of a $7 trillion project could trigger a devastating, worldwide financial depression, far worse than the subprime mortgage crisis of 2007-2008.”
Our reading: The collapse of a $7 trillion AI project could trigger a worldwide financial depression worse than the 2007-2008 crisis.
No horizon
Only if: if the $7 trillion project collapses
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Forecast
“My guess is that is actually a very conservative estimate going forward, particularly if models continue the trend of becoming bigger and more costly to train.”
Our reading: GenAI's future energy needs will probably exceed the power consumption of Germany, especially if models keep growing.
No horizon
Qualifier: particularly if models keep getting bigger and costlier to train
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We should all say “No” to reckless AI expansionism — unless and until tech leaders can show persuasively that the technology is mature enough that such expansion would demonstrably yield enormous net benefit.”
Our reading: We should all say no to reckless AI expansionism unless and until tech leaders can show persuasively that the technology is mature enough that such expansion would demonstrably yield enormous net benefit.
No horizon
Qualifier: unless and until tech leaders can show persuasively that the technology is mature enough that expansion would demonstrably yield enormous net benefit
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Let’s not by any means abandon research in AI, altogether—I do still see great hope if we can get to more trustworthy forms of AI—butfor now we should be focusing primarily on safety and innovative new approaches, rather than rushing to scale up existing and problematic architectures to unfathomable and unpredictable consequences.”
Our reading: AI research should not be abandoned altogether, but for now the focus should be primarily on safety and innovative new approaches rather than rushing to scale up existing, problematic architectures.
No horizon
Qualifier: for now
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“So far GenAI has been mostly promise, rather then delivery. There is no argument yet that it is actually worth tearing up the globe.”
Our reading: So far GenAI has been mostly promise rather than delivery, and there is no argument yet that it is actually worth tearing up the globe.
No horizon
Qualifier: so far
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Observation
“It is bad enough that large language models are making this kind of stuff easy now; worse that companies like OpenAI seem content to let society bear all the costs, like some careless industrial plant belching out toxic chemicals back in the day.”
Our reading: Large language models are now making scams like fake books easy, and companies like OpenAI seem content to let society bear all the costs.
No horizon
Qualifier: seem
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Conditional forecast
“A project that costs $7 trillion is inevitably going to be seen as too too big to fail. Any bailout could easily shatter the world’s economy.”
Our reading: A project that costs $7 trillion will inevitably be seen as too big to fail, and any bailout could easily shatter the world's economy.
No horizon
Only if: a project that costs $7 trillion
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Conditional forecast
“If you want go to fast, fine; if you want to go insanely fast, you risk screwing the pooch, and possibly the entire field of AI along with it.”
Our reading: Scaling AI insanely fast risks failure that could damage the whole field of AI.
No horizon
Only if: if AI expansion goes insanely fast
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Rhetoric
“In asking for $7 trillion, it is pretty obvious to most that Sam Altman has jumped the shark.”
Our reading: Sam Altman's request for $7 trillion shows he has jumped the shark.
No horizon
Source: Seven reasons why the world should say No to Sam Altman
Assessment: Can’t be checkedSaid
Forecast
“My guess is that Altman never gets the money, even in ten yearly $700B tranches.”
Our reading: Altman will probably never raise the $7 trillion, even in ten yearly $700B tranches.
No horizon
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Conditional forecast
“Without massive technical advances, chains of LLMs are unlikely to give us that.”
Our reading: Without massive technical advances, chains of LLMs are unlikely to provide a stable and secure basis for agents.
No horizon
Only if: without massive technical advances
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Observation
“To begin with, it’s just not clear that anywhere near that much capital is available.”
Our reading: It is not clear that anywhere near $7 trillion of capital is available for Altman's project.
No horizon
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But they need to be built on a stable and secure basis.”
Our reading: AI agents need to be built on a stable and secure foundation.
No horizon
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Observation
“Whatever number the investors might run, I am not remotely convinced that we are particularly close to AGI.”
Our reading: Whatever numbers investors might run, Marcus is not remotely convinced that we are particularly close to AGI.
No horizon
Qualifier: whatever number the investors might run
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Rhetoric
“Combine that with worries about security, and it’s all a colossal accident waiting to happen.”
Our reading: Marcus calls device-controlling LLM agents, given their privacy problems combined with worries about security, 'a colossal accident waiting to happen'.
No horizon
Qualifier: privacy access combined with security worries
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Observation
“Nor am I confident that Altmann or anyone else has any idea in the near term about how to make AGI safe.”
Our reading: Marcus is not confident that Altman or anyone else has any idea, in the near term, of how to make AGI safe.
No horizon
Qualifier: in the near term
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Observation
“I won’t say that this can’t happen. But it is deeply speculative.”
Our reading: Marcus will not rule out the upside scenario in which Altman, OpenAI and their partners achieve AGI that conquers sickness and climate change and Altman uses his cut to subsidize everyone else, but he calls that scenario deeply speculative.
No horizon
Qualifier: declines to say it can't happen
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Conditional forecast
“One screwup (and we all know LLMs are prone to screwups) and a LOT of people could die.”
Our reading: If LLM agents ran on Defense Department computers, a single LLM error could kill many people.
No horizon
Only if: if such agents run on Department of Defense computers
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Conditional forecast
“Rogue employees could take over your devices, and bad actors will have tremendous incentive to hack OpenAI’s security.”
Our reading: If OpenAI's software can take over user devices, rogue employees could take over devices and attackers will have strong incentive to hack OpenAI.
No horizon
Only if: if OpenAI's software can take over user devices
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Conditional forecast
“Nor am I convinced that the proceeds (if they did succeed) would be redistributed remotely equitably.”
Our reading: Marcus is not convinced that, if Altman's $7 trillion project succeeded, its proceeds would be redistributed remotely equitably.
No horizon
Only if: if the project succeeded
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In short, I am putting LLM-based agents distributed to millions of customers at the top of my list of most dangerous ideas of 2024.”
Our reading: Marcus puts LLM-based agents distributed to millions of customers at the top of his own list of the most dangerous ideas of 2024.
No horizon
Qualifier: distributed to millions of customers
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The best realistic case is that LLM-based agents never see widespread release. If we are lucky, their likely problems with reliability and all the associated risks will cause them to die on the vine.”
Our reading: The best realistic outcome is that LLM-based agents never see widespread release, dying because of their likely reliability problems.
No horizon
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Observation
“First, in The Information earlier this week was a hint that OpenAI might not be knocking it out of the park economically.”
Our reading: There are signs OpenAI may not be doing very well economically.
No horizon
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Forecast
“I doubt we will have the maturity we need in the next several years.”
Our reading: AI agent technology is unlikely to reach the needed maturity for reliable use in the next several years.
No horizon
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Forecast
“In the very best case, LLM-based agents will change the world for the better. But I don’t see that happening, certainly not this decade.”
Our reading: Marcus does not see LLM-based agents changing the world for the better, which he calls the very best case, and certainly not this decade.
No horizon
Qualifier: in the very best case
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Conditional forecast
“Even if the $7 trillion bet succeeded (on its own terms), it might tear apart society through radically increased unemployment and social inequality or redrawn geopolitics that are hard even to fathom.”
Our reading: Even if the $7 trillion bet succeeded, it might tear apart society through unemployment, inequality or upheaval in geopolitics.
No horizon
Only if: if the $7 trillion bet succeeded on its own terms
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Giving the keys to the castle to an LLM, at our current level of technological readiness, seems to me to be batshit insane.”
Our reading: At current technological readiness, giving LLMs full control of users' systems is reckless and should not be done.
No horizon
Qualifier: at current level of technological readiness
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Conditional forecast
“If they throw LLM-based agents over the fence, we are all in grave danger.”
Our reading: If OpenAI releases LLM-based agents carelessly, everyone will be in grave danger.
No horizon
Only if: if OpenAI releases LLM-based agents without responsibility
Source: What Could Possibly Go Wrong with Sam Altman’s New Ambitions?
Assessment: Can’t be checkedSaid
Observation
“These systems don’t deeply understand their inputs, contra Dr. Hinton, and as pointed out here regularly.”
Our reading: Large language models do not deeply understand their inputs.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We really should consider trying some other approaches to AI.”
Our reading: The AI field should consider approaches to AI other than scaling large language models.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“You shouldn’t count on these systems.”
Our reading: People should not rely on large language model systems.
No horizon
Assessment: Can’t be checkedSaid
Observation
“And they continue to be unstable in virtually everything they do.”
Our reading: Generative AI systems continue to be unstable in virtually everything they do.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“There are many legitimate near-term fears about the misuse of AI, ranging from mis- and disinformation to voice cloning scams to bias to deepfake scams and deepfake porn.”
Our reading: There are many legitimate near-term fears about the misuse of AI, ranging from misinformation and disinformation to voice cloning scams, bias, deepfake scams and deepfake porn.
No horizon
Qualifier: near-term
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Please, developers and military personnel, don’t let your chatbots grow up to generals.”
Our reading: Developers and military personnel should not let chatbots take on the role of military generals.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Letting them run a war would be malpractice.”
Our reading: Generative AI systems should not be allowed to run a war; doing so would be malpractice.
No horizon
Assessment: Can’t be checkedSaid
Observation
“One might conceivably attribute some shallow level of understanding to LLMs, but there are many reasons to doubt that current neural networks have deep understanding.”
Our reading: There are many reasons to doubt that current neural networks have deep understanding, although one might conceivably attribute some shallow level of understanding to LLMs.
No horizon
Scope: current neural networks
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“By not even acknowledging the growing literature on memorization (let alone challenging it), Hinton made himself look to be seriously out of touch.”
Our reading: By not even acknowledging, let alone challenging, the growing literature on memorization, Hinton made himself look seriously out of touch.
No horizon
Qualifier: by not even acknowledging (let alone challenging) the growing literature on memorization
Assessment: Can’t be checkedSaid
Observation
“Now it’s AI developers who are expecting to do what they do scot-free, while society picks up the tab.”
Our reading: AI developers expect to operate without paying for the harms they cause, leaving society to bear the costs, much as polluting factories once did.
No horizon
Source: Deepfaked Shit is Getting Real
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That needs to change.”
Our reading: The situation in which AI developers do what they do scot-free while society picks up the tab for the consequences needs to change.
No horizon
Source: Deepfaked Shit is Getting Real
Assessment: Can’t be checkedSaid
Rhetoric
“Black Mirror has arrived, ahead of schedule.”
Our reading: Dystopian, Black Mirror-style AI harms such as deepfake fraud have already arrived, sooner than expected.
No horizon
Source: Deepfaked Shit is Getting Real
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“but the media and the public need to realize that company white papers are not peer-reviewed articles. Particularly when it comes to safety-critical questions, peer-review is essential.”
Our reading: The media and public must recognize that company white papers are not peer-reviewed, and peer review is essential for safety-critical questions.
No horizon
Qualifier: safety-critical questions
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Forecast
“Unfortunately, in the specific domain in which we are talking about, bioweapons risk, even a tiny effect could literally be world-changing, in a deeply negative way.”
Our reading: In the bioweapons domain, even a tiny effect of LLMs on capability could be world-changing in a deeply negative way.
No horizon
Qualifier: in the bioweapons domain
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Unfortunately footnote [C] is extremely dubious.”
Our reading: Footnote C of OpenAI's paper, which says the results are not statistically significant, is extremely dubious.
No horizon
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If an LLM equips even one team of lunatics with the ability to build, weaponize and distribute even one pathogen as deadly as covid-19, it will be really, really big deal.”
Our reading: If an LLM equips even one team of lunatics with the ability to build, weaponize and distribute even one pathogen as deadly as covid-19, that will be a really, really big deal.
No horizon
Only if: if an LLM equips even one team of lunatics to build, weaponize and distribute even one pathogen as deadly as covid-19
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The threshold itself is arbitrary, and it is rarely good practice to use thresholds in statistical analysis, because doing so throw away information”
Our reading: The 8-of-10 threshold OpenAI used is arbitrary, and using thresholds in statistical analysis is rarely good practice because doing so throws away information.
No horizon
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Conditional forecast
“Statistical significance is in part a function of sample size; with a bigger sample size, I doubt there would be any question about significance at all.”
Our reading: With a bigger sample size, the author doubts there would be any question that the effect of LLM access is statistically significant.
No horizon
Only if: if the sample size were bigger
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Observation
“What the results really seem to be showing is that (a) the nonexperts aren’t so much to be worried about, because few get over the hump, but (b) experts really do seem to be helped, in every aspect of development.”
Our reading: The study results seem to show that nonexperts are not much of a worry because few clear the threshold, but experts do seem to be helped in every aspect of bioweapon development.
No horizon
Qualifier: as interpretation of the study results
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It should go without saying that any risk-benefit analyses that with respect to the tradeoffs involved in AI needs to reflect studies that are not just methodologically sound but also interpreted accurately.”
Our reading: AI risk-benefit analyses need to be based on studies that are both methodologically sound and accurately interpreted.
No horizon
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Causal claim
“Indeed, in a domain like this, thresholds can be deeply misleading, because we don’t really know the relation between a threshold in the experiment and what would matter in the real world.”
Our reading: In the bioweapons domain, experimental thresholds can be deeply misleading because the relation between an experimental threshold and real-world significance is unknown.
No horizon
Qualifier: in the bioweapons domain
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Without question the finding of greater facility by experts on all five tasks deserves followup with a larger group of experts, and a single preregistered test with no Bonferroni correction.”
Our reading: The finding that experts did better on all five tasks should be followed up with a larger group of experts and a single preregistered test without Bonferroni correction.
No horizon
Qualifier: larger group of experts
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Anyone reading this article, OpenAI included, ought to be pretty worried by these results.”
Our reading: Readers, including OpenAI, ought to be pretty worried by the accuracy results in the paper.
No horizon
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“There is zero reason to get much comfort from this threshold analysis.”
Our reading: There is no reason to take much comfort from OpenAI's threshold analysis.
No horizon
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But here there is an obvious a priori test: does using an LLM make people more accurate? That’s what the whole paper is about. You don’t need a Bonferroni correction for that, and shouldn’t be using it.”
Our reading: Because the paper has an obvious a priori hypothesis, a Bonferroni correction is unnecessary and should not have been used.
No horizon
Qualifier: there is an obvious a priori hypothesis
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Good peer reviewers should push not only on sound statistical practice but also on important real-world questions like whether open-source models without guardrails (or with guardrails removed, which is easy to do in open-source models) might be more useful to bad actors than commercial models with guardrails.”
Our reading: Peer reviewers should press on statistical practice and on whether open-source models without guardrails are more useful to bad actors than guardrailed commercial models.
No horizon
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Conditional forecast
“Crucially, if even one malicious expert gets over the hump and creates a deadly pathogen; that’s huge. The results make it look like that sort of thing might be possible.”
Our reading: If even one malicious expert uses an LLM to get over the threshold and create a deadly pathogen that would be hugely consequential, and the study results suggest that might be possible.
No horizon
Only if: if even one malicious expert succeeds
Source: Diving deep into OpenAI’s new study on LLM’s and bioweapons
Assessment: Can’t be checkedSaid
Forecast
“Looks like Google’s brand-new Bard image generation software is going to keep even more IP lawyers gainfully employed this year…”
Our reading: Google's Bard image generation software will likely generate more IP legal work in 2024.
No horizon
Assessment: Can’t be checkedSaid
Observation
“These physical (and in one case conceptual) errors are of a piece with hallucinations and near-plagiarism, too.”
Our reading: The physical errors (and one conceptual error) in the Midjourney image discussed in the post are of a piece with hallucinations and near-plagiarism.
No horizon
Scope: the errors in the Midjourney image discussed in the post
Source: One GenAI image; Ten Errors
Assessment: Can’t be checkedSaid
Observation
“I’ve been worrying about that emphasis on the local at the expense of the global for close to 40 years, and still don’t see a great solution to it.”
Our reading: Marcus says he has been worrying about AI's emphasis on the local at the expense of the global for close to 40 years and still does not see a great solution to it.
No horizon
Qualifier: in his own assessment
Source: One GenAI image; Ten Errors
Assessment: Can’t be checkedSaid
Forecast
“Valuations for OpenAI and some of its peers may drop, sharply.”
Our reading: Valuations of OpenAI and some peers may drop sharply.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Fifth, although the underlying technology initially improved rapidly, leading to a lot of excitement, it may soon, perhaps this year or next, reach a plateau.”
Our reading: LLM technology may reach a plateau, perhaps in 2024 or 2025.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“It would not be surprising if OpenAI spends a huge part of its energy this year defending lawsuits.”
Our reading: OpenAI may spend a huge part of its energy in 2024 defending lawsuits.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Sixth, although ChatGPT has already widely been adopted by computer programmers and perhaps others, and been explored by a great many Chief Information Officers and the like, it is still not clear that there will be enough adoption to justify the company’s stratospheric valuations.”
Our reading: It is unclear that ChatGPT adoption will be enough to justify OpenAI's very high valuation.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“External investors pouring money into the company (most recently at a valuation rumored to be $86 billion dollars) may find themselves disappointed, unclear on how profits to justify those valuations could ever be made.”
Our reading: External investors putting money into OpenAI, most recently at a rumored $86 billion valuation, may find themselves disappointed and unclear on how profits to justify those valuations could ever be made.
No horizon
Qualifier: valuation rumored to be $86 billion
Assessment: Can’t be checkedSaid
Forecast
“Large language models such as ChatGPT may quickly become commodities; we can expect price wars, and profits may continue to be elusive, or modest at best.”
Our reading: Large language models such as ChatGPT may quickly become commodities, price wars can be expected, and profits may continue to be elusive or modest at best.
No horizon
Qualifier: we can expect price wars
Assessment: Can’t be checkedSaid
Forecast
“Regardless of how the internal investigation comes out, 2024 is a looking a lot more challenging, both for Altman and for the company.”
Our reading: 2024 looks likely to be a much more challenging year for OpenAI and Sam Altman.
No horizon
Qualifier: regardless of internal investigation outcome
Assessment: Can’t be checkedSaid
Observation
“The request is likely in violation both of common law and of international conventions around intellectual property such as the Berne convention, and publishers, writers, and artists have every right to fight it tooth and nail.”
Our reading: OpenAI's request for a copyright exemption is likely in violation of common law and international IP conventions like Berne.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“(Regulation in the EU and elsewhere may force OpenAI to confront problems with their software around unreliability, privacy, its off-label use to generate misinformation, and in the service of cybercrime, and so on).”
Our reading: EU and other regulation may force OpenAI to confront unreliability, privacy, misinformation, and cybercrime problems.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Progress depends heavily on having vast amounts of data, and it is likely that most of the high quality data sources have already been tapped.”
Our reading: Most high-quality data sources for LLM training have likely already been tapped.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“First, lawsuits against the company are likely to come fast and furious.”
Our reading: OpenAI will likely face a rapid flurry of lawsuits in 2024.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“It’s also worth noting that OpenAI is not alone is all of this; many other generative AI companies are likely to encounter intense legal and public pressure this year, around potential cases of copyright infringement.”
Our reading: Many generative AI companies are likely to face intense legal and public pressure over copyright in 2024.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“This call could easily put a further strain on the company, and perhaps lead to other uncomfortable revelations if California chooses to investigate.”
Our reading: Public Citizen's call could strain OpenAI and, if California investigates, lead to uncomfortable revelations.
No horizon
Only if: if California chooses to investigate
Assessment: Can’t be checkedSaid
Forecast
“Here’s my own guess: OpenAI, backed by Microsoft’s trillions, and now deeply tied to Microsoft, will survive, but the bloom will fade from the rose.”
Our reading: Marcus guesses that OpenAI, backed by and deeply tied to Microsoft, will survive, but that the bloom will fade from the rose.
No horizon
Qualifier: my own guess
Assessment: Can’t be checkedSaid
Conditional forecast
“If a plateau is reached in the near-term, a lot of the enthusiasm may dissipate, and investors may start to temper their investments.”
Our reading: If LLMs plateau in the near term, enthusiasm may dissipate and investors may temper their investments.
No horizon
Only if: if a plateau is reached in the near-term
Assessment: Can’t be checkedSaid
Forecast
“A lot of what has driven excitement about OpenAI has been a sense of unlimited potential. That could soon change.”
Our reading: The perception of OpenAI's unlimited potential could soon change.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Moreover more data tends to go with bigger models that more expensive to operate; the increases in cost may not be accompanied by big enough gains in performance, at some point leading the big companies to step off the gas.”
Our reading: The rising costs of bigger models trained on more data may not be accompanied by big enough gains in performance, at some point leading the big companies to step off the gas.
No horizon
Scope: bigger models that go with more data are more expensive to operate