Claims that can’t be checked by Gary Marcus, page 20 of 24
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Claims that can’t be checked by Gary Marcus (2353)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 20 of 24: claims 1901 to 2000 of 2353, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Observation
“That projection probably comes from their two best quarters, and some data suggest their spectacular growth has already been a slow down after the amazing initial takeoff.”
Our reading: OpenAI's projected $1 billion in revenue this year probably comes from its two best quarters, and some data suggest its growth has already slowed after the initial takeoff.
No horizon
Qualifier: probably
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: Can’t be checkedSaid
Forecast
“It might be OK in domains where 70% correct is OK, but unlikely to be acceptable in mission-critical applications.”
Our reading: LLMs may be acceptable where 70% accuracy suffices but are unlikely to be acceptable for mission-critical applications.
No horizon
Qualifier: mission-critical applications
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: Can’t be checkedSaid
Forecast
“LLMs will likely never deliver on the promise that they are somehow close to AGI.”
Our reading: LLMs will likely never deliver on the promise of being close to AGI.
No horizon
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: Can’t be checkedSaid
Observation
“In the current state, even small changes to the underlying database might require complete retraining.”
Our reading: Currently, even small changes to an LLM's training data might require complete retraining.
No horizon
Qualifier: small changes to training data
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: Can’t be checkedSaid
Forecast
“Students don’t as rule have a lot of money; they may quickly turn to free alternatives.”
Our reading: Students using OpenAI products may quickly switch to free alternatives.
No horizon
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: Can’t be checkedSaid
Forecast
“Current models still have a lot of problems, necessitating future models. Future models will likely be bigger models, even more expensive to train, even more expensive to operate, further reducing profits.”
Our reading: OpenAI's future models will likely be bigger and more expensive to train and operate, further reducing profits.
No horizon
Source: Top 5 reasons OpenAI is probably not worth 90 billion dollars
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If, after training on virtually the entire internet, you know Tom is Mary Lee‘s son, but can’t figure out without special prompting that Mary Lee therefore is Tom’s mother, you have no business running all the world’s software.”
Our reading: A system that, after training on virtually the entire internet, knows Tom is Mary Lee's son but can't figure out without special prompting that Mary Lee is therefore Tom's mother has no business running all the world's software.
No horizon
Qualifier: after training on virtually the entire internet
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Computers should be expected to make optimal use of information given, regardless of human limitations.”
Our reading: Computers should be expected to make optimal use of the information given, regardless of human limitations.
No horizon
Qualifier: regardless of human limitations
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: Can’t be checkedSaid
Causal claim
“The will to believe in neural networks is frequently so strong that counterevidence is often dismissed or ignored, for much too long.”
Our reading: The will to believe in neural networks is frequently so strong that counterevidence is often dismissed or ignored for much too long.
No horizon
Qualifier: frequently
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: Can’t be checkedSaid
Observation
“None of what I raised then has really been adequately addressed in the intervening decades.”
Our reading: The generalization problems Marcus raised in 2001 about neural networks have not been adequately addressed since.
No horizon
Source: Elegant and powerful new result that seriously undermines large language models
Assessment: Can’t be checkedSaid
Causal claim
“My own guess is that between the release of GPT-4 in November 2022 and the May 2023 testimony OpenAI ran some preliminary tests and decided that the system would not meet expectations.”
Our reading: Marcus guesses that, between GPT-4's release (which he dates to November 2022) and the May 2023 testimony, OpenAI ran some preliminary tests and decided GPT-5 would not meet expectations, and that this explains why it is not training GPT-5.
No horizon
Qualifier: my own guess
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We have far too little machinery in place to surveil or address such threats.”
Our reading: We have far too little machinery in place to surveil or address threats such as a new form of bioweapon attack that could arise if superintelligent AI were at some point imminent and posed a genuine threat.
No horizon
Scope: threats from hypothetically imminent superintelligent AI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But here’s someone else the AI industry is taking advantage of: you.”
Our reading: The AI industry is taking advantage of ordinary people, in this context through harvesting their personal data.
No horizon
Scope: in the context of collecting personal data (Meta, Google, Zoom, carmakers)
Source: The dirty underbelly of AI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“My own view is that Cyc has been neither a success nor a failure, but somewhere in between: I see it as a ground-breaking, clarion experiment that never fully gelled.”
Our reading: In Marcus's own view, Cyc has been neither a success nor a failure but somewhere in between: a ground-breaking, clarion experiment that never fully gelled.
No horizon
Qualifier: in his own view
Source: Doug Lenat, 1950-2023
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“He didn’t fully succeed – we will need a different path – but he picked the critical mountain that we still must cross.”
Our reading: Capturing common sense is the critical problem AI still has to solve, but it will take a different path from Cyc's.
No horizon
Source: Doug Lenat, 1950-2023
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But every single one of them should know something about Cyc.”
Our reading: Every young AI researcher should know something about Cyc.
No horizon
Source: Doug Lenat, 1950-2023
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Not because Cyc will get used out of the box, as some sort of drop-in replacement for Large Language Models, but because what Lenat tried to do - to get machines to represent and reason about common sense — still must be done.”
Our reading: Getting machines to represent and reason about common sense still has to be done, though Cyc itself will not serve as a drop-in replacement for LLMs.
No horizon
Source: Doug Lenat, 1950-2023
Assessment: Can’t be checkedSaid
Conditional forecast
“No, Cyc didn’t set the world on fire, but yes, it will seem more and more important in hindsight, as we eventually make real progress towards artificial general intelligence.”
Our reading: Cyc will seem more and more important in hindsight as real progress toward artificial general intelligence is eventually made.
No horizon
Only if: as real progress toward artificial general intelligence is eventually made
Source: Doug Lenat, 1950-2023
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The biggest problem with Cyc from an academic perspective is that it’s proprietary.”
Our reading: From an academic perspective, Cyc's biggest problem is that it is proprietary.
No horizon
Qualifier: from an academic perspective
Source: Doug Lenat, 1950-2023
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If we are to get the right regulatory regime for AI – one that protects consumers, without stifling innovation – we are going to need government to step up.”
Our reading: Government must step up to achieve an AI regulatory regime that protects consumers without stifling innovation.
No horizon
Qualifier: to get the right regulatory regime for AI
Source: Could we all be doomed, without ranked-choice voting?
Assessment: Can’t be checkedSaid
Forecast
“I rather doubt that will happen, but I am not sure that it won’t.”
Our reading: An AI-induced extinction event that kills everyone is unlikely, though not ruled out.
No horizon
Source: Could we all be doomed, without ranked-choice voting?
Assessment: Can’t be checkedSaid
Observation
“The goal of the companies is return to shareholders, not to return to humanity.”
Our reading: AI companies aim at returns to shareholders rather than benefit to humanity.
No horizon
Source: Could we all be doomed, without ranked-choice voting?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But I think ranked-choice-voting (or some similar alternative) is very much worth serious consideration.”
Our reading: Marcus thinks ranked-choice voting, or some similar alternative, is very much worth serious consideration.
No horizon
Qualifier: I think
Source: Could we all be doomed, without ranked-choice voting?
Assessment: Can’t be checkedSaid
Conditional forecast
“What I am sure of, though, is that our chances to avoid serious consequences aren’t great if we control neither AI nor the companies that make AI.”
Our reading: If neither AI nor AI companies are controlled, society's chances of avoiding serious consequences are poor.
No horizon
Only if: if we control neither AI nor the companies that make AI
Source: Could we all be doomed, without ranked-choice voting?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We waited too long on climate change; to the extent that there may be any real risk, either of outright doom or “mere” catastrophe, let’s not wait too long on AI.”
Our reading: To the extent AI carries any real risk of doom or catastrophe, society should not wait too long to address it, unlike with climate change.
No horizon
Qualifier: to the extent there may be real risk of doom or catastrophe
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Observation
“I am, at least for now, much more worried about the overapplication of mediocre AI (eg what happens if we put large language models into running military strategy or into hiring decisions that affect millions of people) than the use of AI that might be so smart it could catch its own errors.”
Our reading: Marcus says that, at least for now, he is much more worried about the overapplication of mediocre AI (e.g., putting large language models into running military strategy or into hiring decisions affecting millions) than about the use of AI that might be so smart it could catch its own errors.
No horizon
Qualifier: at least for now
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Forecast
“And what really gets the goat of people like Timnit Gebru and Abeba Birhane, not unreasonably, is that there is a good chance that any immediate effects will disproprotionately affect the dispossessed; they are very likely right about that.”
Our reading: Marcus says Gebru and Birhane are very likely right that there is a good chance any immediate effects of AI will disproportionately affect the dispossessed.
No horizon
Qualifier: a good chance
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Unlike Yudkowsky I don’t think the probabilities are high enough—or imminent enough- to warrant bombing data centers any time soon. (Or probably ever.)”
Our reading: AI doom probabilities are not high or imminent enough to justify bombing data centers any time soon, and probably never will be.
No horizon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It’s therefore worth getting a rough idea of the number, even if we can’t achieve consensus on the exact number.”
Our reading: It is worth getting a rough estimate of p(doom) even if there is no consensus on the exact number.
No horizon
Qualifier: even without consensus on the exact number
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Observation
“None of this means of course that I suddenly think that we are close to super-smart machines. (Spoiler alert: we are not).”
Our reading: We are not close to super-smart machines.
No horizon Evidence as of
Limitation: This is not a finding that superintelligence was near or far in 2023.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“no matter what you feel about that p(doom | superintelligence), whether you see it as sci-fi or not, you should be worried about p(catastrophe | widespread AI adoption)”
Our reading: Whatever one thinks about doom from superintelligence, people should worry about the probability of catastrophe given widespread AI adoption.
No horizon
Qualifier: regardless of views on superintelligence doom
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Forecast
“Perhaps extinction is vanishingly unlikely, but what I will call 𝐩(𝐜𝐚𝐭𝐚𝐬𝐭𝐫𝐨𝐩𝐡𝐞) – the chance that an incident that kills (say) one percent or more of the population, seems a lot higher.”
Our reading: Perhaps extinction is vanishingly unlikely, but the chance of an incident that kills, say, one percent or more of the population seems a lot higher.
No horizon
Qualifier: perhaps
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Causal claim
“As Englander well knows, we will never get perfect estimates, but even imperfect estimates might help guide policy decisions as we try to balance attention to short-term and long-term risks in AI.”
Our reading: We will never get perfect estimates of p(doom) or p(catastrophe), but even imperfect estimates might help guide policy decisions as we try to balance attention to short-term and long-term AI risks.
No horizon
Qualifier: even imperfect estimates
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Observation
“LeCun’s conclusion —that humans will always set the agenda—might be true, but he hasn’t given a real argument.”
Our reading: LeCun has given no real argument that humans will always set the agenda over AI, though that conclusion might be true.
No horizon Evidence as of
Limitation: This does not endorse LeCun’s conclusion or establish that his argument is sound.
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Forecast
“It’s a story for another day, but clearly you can have catastrophe without mass death, and we are heading in that direction.”
Our reading: AI can bring catastrophe without mass death, and society is heading in that direction.
No horizon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Observation
“There are in fact already some scenarios that are faintly imaginable now (e.g. bad actors using machines to create novel toxins, in ways that go awry), and over time could be more.”
Our reading: Some faintly imaginable scenarios already exist in which machines could kill everyone, such as bad actors using machines to create novel toxins, and more such scenarios could appear over time.
No horizon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But imprecision is not an excuse for inaction.”
Our reading: Imprecise risk estimates do not justify failing to act on AI risk.
No horizon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Unfortunately, the fact that we can’t predict the course of technology with any precision even a decade out doesn’t magically immunize us from its effects, nor relieve us of a moral responsbility to try to take precautions.”
Our reading: Not being able to predict technology precisely does not excuse us from the moral responsibility to take precautions against its risks.
No horizon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Observation
“But bluntly, uncontrollable superintelligence is by no means the only risk from AI, and from a policy perspective may not be the most important risk, given both the lowish (?) probability and the time horizon.”
Our reading: Uncontrollable superintelligence is by no means the only risk from AI and, from a policy perspective, may not be the most important risk, given both its lowish (?) probability and the time horizon.
No horizon
Qualifier: from a policy perspective
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Forecast
“The risks of current AI (bias, defamation, cybercrime, wholesale disinformation, etc) are already starting to be well documented and may themselves quickly escalate, and could lead to geopolitical instability even over the next few years.”
Our reading: Documented risks of current AI such as bias, defamation, cybercrime and disinformation may escalate quickly and could cause geopolitical instability within the next few years.
No horizon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Observation
“Personally, despite the name of this essay, I am not all that worried about humans going extinct anytime very soon.”
Our reading: Marcus says he is not all that worried about humans going extinct anytime very soon.
No horizon
Scope: anytime very soon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Forecast
“I do think there is a lot of risk that AI will cause chaos.”
Our reading: There is a substantial risk that AI will cause chaos.
No horizon
Source: 𝐩(𝐝𝐨𝐨𝐦)
Assessment: Can’t be checkedSaid
Observation
“The fact that the US is apparently using these tools, or the fact that they aren’t likely to be reliable, almost certainly regularly reporting false alarms, and likely full of insidious bias, like so many previous AI systems have shown themselves to be?”
Our reading: The AI surveillance tools the US government is apparently using are likely not reliable.
No horizon
Assessment: Can’t be checkedSaid
Observation
“The fact that the US is apparently using these tools, or the fact that they aren’t likely to be reliable, almost certainly regularly reporting false alarms, and likely full of insidious bias, like so many previous AI systems have shown themselves to be?”
Our reading: The AI tools the US is apparently using to surveil its citizens are likely full of insidious bias, as so many previous AI systems have shown themselves to be.
No horizon
Qualifier: likely
Assessment: Can’t be checkedSaid
Causal claim
“This could be particularly devastating because Large Language Models aren’t like classical databases in which individual pieces of data can be removed at will; if any content is removed, the entire model must (so far as I understand it) be retrained, at great expense.”
Our reading: The potential New York Times lawsuit could be particularly devastating for OpenAI because, unlike classical databases, LLMs cannot have individual pieces of data removed at will; as far as Marcus understands it, removing any content means retraining the entire model at great expense.
No horizon
Only if: if any content is removed
Source: The Rise and Fall of ChatGPT?
Assessment: Can’t be checkedSaid
Rhetoric
“It ain’t over yet, but what a stunning reversal of fortune.”
Our reading: ChatGPT has suffered a stunning reversal of fortune, though the story is not over.
No horizon
Qualifier: it ain't over yet
Source: The Rise and Fall of ChatGPT?
Assessment: Can’t be checkedSaid
Observation
“Maybe it’s too soon to declare victory on my August 12th bet that ChatGPT will turn out to be a dud, but I will tell you one thing, it’s not obvious I am wrong.”
Our reading: As of late August 2023, it is not obvious that Marcus's bet that ChatGPT will turn out to be a dud is wrong, though it may be too soon to declare victory.
No horizon
Qualifier: may be too soon to declare victory
Source: The Rise and Fall of ChatGPT?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We see few risks in the current regime, but fewer reasons to tempt fate by blithely assuming that anything that we might invent can be dealt with.”
Our reading: We should not blithely assume that anything we might invent in AI can be dealt with.
No horizon
Source: Sentient AI: For the love of Darwin, let’s Stop to Think if We Should
Assessment: Can’t be checkedSaid
Causal claim
“AI, like any technology, is subject to the risk of unintended consequences, quite possibly more so, and the wider we open Pandora’s box, the more risk we assume.”
Our reading: AI is subject to the risk of unintended consequences, possibly more than other technologies, and expanding its capabilities increases that risk.
No horizon
Qualifier: the wider we open Pandora's box
Source: Sentient AI: For the love of Darwin, let’s Stop to Think if We Should
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Let’s not dumb down our standards, and wind up with a bunch of machines we can’t rely on, just so that big tech can prematurely declare victory, when there is plenty of work left to do.”
Our reading: AGI standards should not be lowered so big tech can prematurely declare victory, since much work remains.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“I seriously doubt it.”
Our reading: Marcus seriously doubts that AI is close to beating typical human performance on virtually all cognitive tasks.
No horizon
Qualifier: AGI as defined by Legg
Assessment: Can’t be checkedSaid
Observation
“Humans can write summaries and reliably answer many questions about original films and television programs that they watch; I doubt LLMs (or any other extant AI technology) can do this at all.”
Our reading: Marcus doubts LLMs or any extant AI can summarize and reliably answer questions about original films and TV programs.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It’s a huge downgrade in our goals for artificial general intelligence to say “better than the average untrained human on many tasks”; the point is that we should expect machines to do what we ask them to do;”
Our reading: Saying AGI means 'better than the average untrained human on many tasks' is a huge downgrade in our goals for AGI, because we should expect machines to do what we ask them to do.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“An artificial general intelligence should do the things that it is asked to do, and decline to do those that it can’t, and have the wisdom to know the difference.”
Our reading: An AGI should do what it is asked, decline what it cannot do, and know the difference.
No horizon
Assessment: Can’t be checkedSaid
Observation
“And frankly, we still have a long way to go on all the things I always harp on: reasoning, compositionality, planning, factuality, and so on.”
Our reading: AI still has a long way to go on reasoning, compositionality, planning, and factuality.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“No AGI worth its salt would brick multidigit arithmetic.”
Our reading: A genuine AGI must reliably do multidigit arithmetic.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Dropping from “as smart as the Star Trek computer” to better than Joe Sixpack should not be our goal.”
Our reading: The AI field should not lower its AGI goal from Star Trek computer-level to merely beating an average person.
No horizon
Assessment: Can’t be checkedSaid
Rhetoric
“(If that doesn’t remind you of LLMs, you aren’t paying attention.)”
Our reading: Marcus suggests that the way driverless-car safety depends on data rather than reasoning should remind attentive readers of LLMs.
No horizon
Source: Face it, self-driving cars still haven’t earned their stripes
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Scaling up to driving everywhere all the time without a well-vetted serious, well-vetted solution to the edge case problem was insane; it was quite literally an accident (or series of accidents) waiting to happen.”
Our reading: Approving 24/7 driverless operation across San Francisco without a vetted solution to edge cases was reckless and bound to lead to accidents.
No horizon
Qualifier: without a well-vetted solution to edge cases
Source: Face it, self-driving cars still haven’t earned their stripes
Assessment: Can’t be checkedSaid
Conditional forecast
“If we don’t, we are likely to see reprises of what we see now with driverless cars, in automated doctors, automated psychiatrists, all purpose virtual assistants, home robots, and more, probably for years to come.”
Our reading: If we do not shift our focus toward developing new methodologies for coping with edge cases, we are likely to see reprises of today's driverless-car problems in automated doctors, automated psychiatrists, all-purpose virtual assistants, home robots and more, probably for years to come.
No horizon
Only if: if we don't stop hammering a square peg into a round hole and shift focus toward developing new methodologies for edge cases
Source: Face it, self-driving cars still haven’t earned their stripes
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I wouldn’t trust a self-driving plane, and I don’t think any quasi-self-driving car has yet earned its stripes either.”
Our reading: Marcus would not trust a self-driving plane, and he does not think any quasi-self-driving car has yet earned its stripes.
No horizon
Qualifier: I don't think
Source: Face it, self-driving cars still haven’t earned their stripes
Assessment: Can’t be checkedSaid
Forecast
“Elections may hinge on them.”
Our reading: Elections may hinge on automatic misinformation generation systems.
No horizon
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We ought to demand that any content like this that is automatically generated be labeled (no existing law does that) and watermarked (no existing law does that either)”
Our reading: We ought to demand that any automatically generated content like this (the CounterCloud-style misinformation discussed) be labeled and watermarked, and no existing law requires either.
No horizon
Scope: any content like this that is automatically generated
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: Can’t be checkedSaid
Forecast
“The temptation for all sides to use these tools is likely to be overwhelming.”
Our reading: The temptation for all political sides to use automatic misinformation tools is likely to be overwhelming.
No horizon
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“What we really need is AI that is smart enough to know when it is being used to spew falsehood.”
Our reading: We need AI smart enough to know when it is being used to spread falsehoods.
No horizon
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: Can’t be checkedSaid
Rhetoric
“The killer app of AGI may be killing truth.”
Our reading: The killer app of AGI may be killing truth.
No horizon
Qualifier: may
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“we ought to demand that the wholesale generation of misinformation be punishable; there ought to be enforcement.”
Our reading: Wholesale generation of misinformation should be punishable and enforced against.
No horizon
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: Can’t be checkedSaid
Conditional forecast
“Businesses may be at a disadvantage if they too don’t use such systems, e.g., to counter adverse news, automatically.”
Our reading: Businesses that do not use automatic misinformation generation systems may be at a disadvantage.
No horizon
Only if: if they don't use automatic misinformation generation systems
Source: Automatic Disinformation Threatens Democracy— and It’s Here
Assessment: Can’t be checkedSaid
Forecast
“Cesspools of automatically-generated fake websites, rather than ChatGPT search, may ultimately come to be the single biggest threat that Google ever faces.”
Our reading: Automatically generated fake websites, rather than ChatGPT search, may eventually become the biggest threat Google ever faces.
No horizon
Assessment: Can’t be checkedSaid
Observation
“But things have actually gotten worse than I anticipated.”
Our reading: Pollution of the internet by LLM-generated content became worse between February and August 2023 than Marcus had expected.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“After all, if users are left sifting through sewers full of useless misinformation, the value of search would go to zero—potentially killing the company.”
Our reading: If users are left sifting through floods of useless misinformation, the value of search would fall to zero, which could kill Google.
No horizon
Only if: if users are left sifting through sewers full of useless misinformation
Assessment: Can’t be checkedSaid
Observation
“The astronomical valuations for Generative AI companies might be justified, but might well not be. Thus far, the valuations seem to be predicated on hopes and dreams, without really factoring in the serious engineering risks.”
Our reading: The astronomical valuations of generative AI companies might be justified but might well not be, and so far they seem to be predicated on hopes and dreams without really factoring in the serious engineering risks.
No horizon
Qualifier: might be justified, but might well not be
Assessment: Can’t be checkedSaid
Forecast
“A lot of talent and investors might move on to a new set of shiny things, fleeing generative AI as quickly as some of the same people not long ago fled crypto.”
Our reading: A lot of talent and investors might move on to a new set of shiny things, fleeing generative AI as quickly as some of the same people recently fled crypto.
No horizon
Scope: a lot of talent and investors
Assessment: Can’t be checkedSaid
Forecast
“As I argued yesterday, the whole thing could turn out to be a fad, but it’s still way too early too tell.”
Our reading: Generative AI could turn out to be a fad, though it is too early to tell.
No horizon
Qualifier: too early to tell
Assessment: Can’t be checkedSaid
Forecast
“I am doubtful that Microsoft would let OpenAI go under anytime soon, given the optics, and impact on their own stock price.”
Our reading: Microsoft is unlikely to let OpenAI go under anytime soon.
No horizon
Qualifier: given optics and stock price impact
Assessment: Can’t be checkedSaid
Observation
“It’s not a house of cards, but the financial foundation of so-called Foundation Models is hardly as robust as it seems.”
Our reading: The financial foundation of so-called foundation models is not a house of cards, but it is hardly as robust as it seems.
No horizon
Qualifier: not a house of cards
Assessment: Can’t be checkedSaid
Causal claim
“We now know that answers can even change from one month to the next—which makes third-party engineering of large language models into complex systems a huge uphill struggle. In some mission-critical cases that engineering challenge may be insuperable.”
Our reading: Because LLM answers can even change from one month to the next, third-party engineering of large language models into complex systems is a huge uphill struggle, and in some mission-critical cases that challenge may be insuperable.
No horizon
Qualifier: answers can change from one month to the next
Assessment: Can’t be checkedSaid
Conditional forecast
“If some people get off the bus, and valuations begin to drop a little, that could launch a potent negative feedback spiral, sparking a sudden deceleration in values that heretofore had been rapidly accelerating.”
Our reading: If some people get off the bus and generative AI valuations begin to drop a little, that could launch a potent negative feedback spiral, sparking a sudden deceleration in values that had been rapidly accelerating.
No horizon
Only if: if some people get off the bus and valuations begin to drop a little
Assessment: Can’t be checkedSaid
Forecast
“LLMs will probably get cheaper to operate over time, both as the hardware required inevitably gets cheaper, and as people figure out how to make them more efficient.”
Our reading: LLMs will probably become cheaper to operate over time as hardware cheapens and efficiency improves.
No horizon
Qualifier: hardware gets cheaper
Assessment: Can’t be checkedSaid
Observation
“That said, I don’t think OpenAI is actually in any kind of imminent danger, but nor do I think they are out of the woods.”
Our reading: Marcus does not think OpenAI is in any kind of imminent danger, but he also does not think it is out of the woods.
No horizon Evidence as of
Limitation: This does not judge the reasonableness of his financial concerns.
Assessment: Can’t be checkedSaid
Forecast
“I seriously doubt it. We have not one, but many, serious, unsolved problems at the core of generative AI”
Our reading: Marcus seriously doubts the projections of near-term AGI, including Dario Amodei's projection that we will have AGI in 2-3 years from August 2023.
Deadline passed Evidence as of
Limitation: This record does not assess whether AGI arrived by August 2026. The post describes AGI broadly, as general purpose AI 'as smart and resourceful as humans if not more so', but gives no test for it.
Assessment: Can’t be checkedSaid
Observation
“But neither coding nor high-speed, mediocre quality copy-writing are remotely enough to maintain current valuation dreams.”
Our reading: Coding and mediocre copywriting use cases are nowhere near enough to justify current generative AI valuations.
No horizon Evidence as of
Limitation: The evidence does not establish that the valuation judgment was right or wrong.
Assessment: Can’t be checkedSaid
Conditional forecast
“If hallucinations aren’t fixable, generative AI probably isn’t going to make a trillion dollars a year. And if it probably isn’t going to make a trillion dollars a year, it probably isn’t going to have the impact people seem to be expecting.”
Our reading: If hallucinations aren't fixable, generative AI probably won't earn a trillion dollars a year or have the expected impact.
No horizon
Only if: if hallucinations aren't fixable
Assessment: Can’t be checkedSaid
Observation
“With the possible exception of the quick to rise and quick to fall alleged room-temperature superconductor LK-99, few things I have ever seen have been more hyped than generative AI.”
Our reading: Generative AI is among the most hyped things the author has ever seen.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“And if it isn’t going to have that impact, maybe we should not be building our world around the premise that it is.”
Our reading: If generative AI isn't going to have the impact people seem to expect, maybe we should not be building our world around the premise that it will.
No horizon
Only if: if generative AI isn't going to have the impact people seem to be expecting
Assessment: Can’t be checkedSaid
Forecast
“Even OpenAI could have a hard time following through on its $29 billion valuation; competing startups valued in the low billions might well eventually collapse, if year after year they manage only tens or hundreds of millions in revenue.”
Our reading: OpenAI could have difficulty living up to its $29 billion valuation.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“Even OpenAI could have a hard time following through on its $29 billion valuation; competing startups valued in the low billions might well eventually collapse, if year after year they manage only tens or hundreds of millions in revenue.”
Our reading: Generative AI startups valued in the low billions might well eventually collapse if they earn only tens or hundreds of millions annually.
No horizon
Only if: if they manage only tens or hundreds of millions in revenue year after year
Assessment: Can’t be checkedSaid
Conditional forecast
“If it isn’t, the bubble could easily burst.”
Our reading: If hallucination is not solved soon, the generative AI bubble could easily burst.
No horizon
Only if: if hallucination is not solved soon
Assessment: Can’t be checkedSaid
Conditional forecast
“If Evans’ experience is a canary in a coal mine, the whole generative AI field, at least at current valuations, could come to a fairly swift end.”
Our reading: If user disillusionment like Evans' is representative, the generative AI field at current valuations could end fairly swiftly.
No horizon
Only if: if Evans' disillusionment is representative
Assessment: Can’t be checkedSaid
Conditional forecast
“if the premise that generative AI is going to be bigger than fire and electricity turns out to be mistaken, or at least doesn’t bear out in the next decade, it’s certainly possible that we could wind up with what in hindsight is a lot of needless extra tension with China, possibly even a war in Taiwan, over a mirage”
Our reading: If the premise that generative AI will be bigger than fire and electricity proves mistaken, or at least does not bear out in the next decade, it is certainly possible that we could end up with a lot of needless extra tension with China over a mirage, possibly even a war in Taiwan.
No horizon
Only if: if the premise that generative AI will be bigger than fire and electricity turns out to be mistaken, or at least doesn't bear out in the next decade
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Those revenues genuinely could grow by 1000x, but that’s mighty speculative. We shouldn’t simply assume it.”
Our reading: We should not assume generative AI revenues will grow 1000x, which is highly speculative.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Other potential paying customers may lose heart quickly.”
Our reading: Other potential paying customers of generative AI, beyond coders and undergraduates, may lose heart quickly.
No horizon
Scope: other potential paying customers (beyond coders and undergraduates)
Assessment: Can’t be checkedSaid
Forecast
“Microsoft, up for the year by nearly half, perhaps largely on the promise of generative AI, might see a stock slump; NVIDIA skyrocketing even more, might also fall.”
Our reading: Microsoft's stock, up nearly half for the year perhaps largely on the promise of generative AI, might slump, and NVIDIA's stock, which has risen even more, might also fall.
No horizon
Qualifier: perhaps largely on the promise of generative AI
Assessment: Can’t be checkedSaid
Causal claim
“Everybody in industry would probably like you to believe that AGI is imminent. It stokes their narrative of inevitability, and it drives their stock prices and startup valuations.”
Our reading: Everybody in industry would probably like you to believe that AGI is imminent, because that belief stokes their narrative of inevitability and drives their stock prices and startup valuations.
No horizon
Qualifier: probably
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It’s foolish to imagine that such challenging problems will all suddenly be solved.”
Our reading: It is foolish to imagine that generative AI's challenging core problems will all suddenly be solved.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“We might not get the consumer protections we need, because we are trying to foster something that may not grow as expected.”
Our reading: The US might fail to get needed consumer protections because it is fostering generative AI, which may not grow as expected.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“But, to begin with, the revenue isn’t there yet, and might never come.”
Our reading: Generative AI revenue is not yet there and might never materialize.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Throughout the time in which I have been advocating for international AI governance, I have emphasized the need to have scientists and civil society more generally at the table.”
Our reading: International AI governance should include scientists and civil society at the table.
No horizon
Source: Jumpstarting AI Governance
Assessment: Can’t be checkedSaid
Forecast
“Many of us have also become worried about the possibility of regulatory capture, and whether shrewd executives at the major tech companies might induce governments to alight on regulation that ultimately does more harm than good, keeping out smaller players, and entrenching Big Tech’s own status, while doing little to address the underlying risks, such as around misinformation and cybercrime.”
Our reading: Major tech company executives might get governments to adopt AI regulation that does more harm than good by keeping out smaller players and entrenching Big Tech, while doing little about risks like misinformation and cybercrime.
No horizon
Source: Jumpstarting AI Governance
Assessment: Can’t be checkedSaid
Conditional forecast
“If we can produce the right package of tools and expertise for auditing, prerelease examination, and so on, address questions of bias, reliability, misinformation and so forth, working together of course with existing agencies that have been aiming to craft related standards, adoption may be swift.”
Our reading: If CATAI can produce the right package of tools and expertise for auditing, prerelease examination, and so on, and address questions of bias, reliability, misinformation and so forth, working with existing agencies that have been aiming to craft related standards, adoption may be swift.
No horizon
Only if: if CATAI can produce the right package of tools and expertise for auditing, prerelease examination, and so on
Source: Jumpstarting AI Governance