Claims that can’t be checked by Gary Marcus, page 16 of 24
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Claims that can’t be checked by Gary Marcus (2353)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 16 of 24: claims 1501 to 1600 of 2353, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“stop funding a losing hand to the infinite degree, crowding everything out.”
Our reading: We must stop funding a losing hand to the infinite degree, which crowds everything else out.
No horizon
Qualifier: to the infinite degree
Assessment: Can’t be checkedSaid
Forecast
“a pile of unreliable systems that are unlikely to live up to the hype.”
Our reading: Current generative AI systems are unreliable and are unlikely to live up to the hype.
No horizon
Assessment: Can’t be checkedSaid
Observation
“All this happened upon the realization that their systems simply cannot succeed without even more data than the internet-scale data they have already been trained on.”
Our reading: The generative AI systems of major AI companies cannot succeed without even more data than the internet-scale data they were already trained on.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“Mixing bogus chatbot stories in with the rest is a recipe for disaster, both for humans and for LLMs. It’s a downward spiral waiting to happen.”
Our reading: Mixing bogus chatbot stories in with the rest is a recipe for disaster for both humans and LLMs, a downward spiral waiting to happen.
No horizon
Scope: both for humans and for LLMs
Assessment: Can’t be checkedSaid
Causal claim
“Gary Marcus doesn’t know how to it say it more clearly: fast, cheap, automated misinformation left unchecked threatens to undermine trust — and democracy itself.”
Our reading: Unchecked fast, cheap, automated misinformation threatens to undermine trust and democracy.
No horizon
Qualifier: left unchecked
Assessment: Can’t be checkedSaid
Forecast
“Bluntly, we risk falling directly into Putin’s hands:”
Our reading: We risk falling directly into Putin's hands.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Senator Schumer, please take note. We need to move on this, fast.”
Our reading: Congress needs to act quickly on AI misinformation, including labeling requirements.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“One of the (many) suggestions, almost the least we could ask for, is that we should require all AI-generated content to be labeled.”
Our reading: US law should require all AI-generated content to be labeled.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That would be fine if LLMs were kept in the lab, where they arguably still belong”
Our reading: LLMs arguably still belong in the lab rather than in widespread deployment.
No horizon
Qualifier: arguably
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Forecast
“Some get patched, but there very likely will aways be more.”
Our reading: Even as some jailbreaks are patched, there will very likely always be more jailbreak attacks on LLMs.
No horizon
Qualifier: even though some get patched
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That would be fine if LLMs were kept in the lab, where they arguably still belong, but with hundreds of millions of people using them daoly, and companies like OpenAI promising to put them in all purpose agents with essentially full device-level access, the lack of any kind of guarantee gets more worrisome by the day.”
Our reading: With hundreds of millions of people using LLMs daily and companies like OpenAI promising to put them in all-purpose agents with essentially full device-level access, the lack of any kind of guarantee about LLMs gets more worrisome by the day, though it would be fine if LLMs were kept in the lab, where they arguably still belong.
No horizon
Qualifier: hundreds of millions of people using LLMs daily
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Observation
“There seems to be an endless number of ways of attacking them.”
Our reading: There appears to be an endless number of ways to attack LLMs.
No horizon
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Observation
“As noted above, Rule 1 of cybersecurity is to keep your attack surface small; in LLMs the attack surface appears to be infinite. That can’t be good.”
Our reading: The attack surface of LLMs appears to be infinite, violating the cybersecurity principle of keeping attack surfaces small.
No horizon
Source: An unending array of jailbreaking attacks could be the death of LLMs
Assessment: Can’t be checkedSaid
Forecast
“If hallucinations were brought down to human expert levels by the end of 2024 I would be truly astonished.”
Our reading: By the end of 2024, hallucinations by large language models will not have been reduced to the level of human experts.
Deadline passed Evidence as of
Limitation: This record does not show how often AI systems or human experts made factual errors in 2024.
Assessment: Can’t be checkedSaid
Conditional forecast
“If OpenAI manages to put out a truly mindblowing GPT-5 this year, the hype will further increase; the bubble won’t burst soon (even if the use cases to justify it aren’t really there).”
Our reading: If OpenAI releases a truly mindblowing GPT-5 in 2024, hype will increase and the GenAI bubble will not burst soon.
Deadline passed Evidence as of
Only if: if OpenAI releases a truly mindblowing GPT-5 this year
Assessment: Can’t be checkedSaid
Conditional forecast
“If nobody (OpenAI, Google, or anyone else) releases a true quantum leap by the end of 2024, substantially addressing key issues around reliability, hallucination, data leakage, and security, the bubble may start to pop by this time next year.”
Our reading: If no one releases a quantum leap addressing reliability, hallucination, data leakage and security by end of 2024, the GenAI bubble may start to pop by March 2025.
No horizon
Only if: if nobody releases a true quantum leap by end of 2024 addressing reliability, hallucination, data leakage and security
Assessment: Can’t be checkedSaid
Forecast
“To be honest, I am not confident Generative AI as currently envisioned will ever come together.”
Our reading: Generative AI as currently envisioned may never come together.
No horizon
Qualifier: as currently envisioned
Assessment: Can’t be checkedSaid
Conditional forecast
“GenAI will still exist, and find some use, but valuations and excitement may dissipate, reemerging only once genuine advances are made.”
Our reading: GenAI will persist and find some use, but valuations and excitement may dissipate until genuine advances are made.
No horizon
Only if: if no quantum leap by end of 2024
Assessment: Can’t be checkedSaid
Observation
“The entire industry is based on hype, and on the specific hope that the kinds of problems we saw again and again with GPT-2, GPT-3, and GPT-4, not to mention Claude, Grok, Gemini, LLaMA, and so on, are on the verge of being solved — even as the reality is that all the big players are converging on systems”
Our reading: The generative AI industry is based on hype and the hope that persistent LLM problems are about to be solved.
No horizon
Assessment: Can’t be checkedSaid
Observation
“But as time goes on, I increasingly believe that call was correct.”
Our reading: Marcus increasingly believes his August 2023 call that generative AI may turn out to be a dud was correct, though he is not yet claiming victory.
No horizon
Qualifier: not yet claiming victory
Assessment: Can’t be checkedSaid
Conditional forecast
“GenAI will, in that case, live for another day, perhaps imploding only later when people realize there is no killer app to justify the increasingly high costs.”
Our reading: Even with a mindblowing GPT-5 in 2024, GenAI might implode later once people realize there is no killer app to justify rising costs.
No horizon
Only if: if OpenAI releases a truly mindblowing GPT-5 this year
Assessment: Can’t be checkedSaid
Forecast
“GPT-5 may be more satisfying but is likely even more expensive to train and operate.”
Our reading: GPT-5 may be more satisfying but is likely even more expensive to train and operate than current models.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“But will we see a mindblowing GPT this year? I doubt it.”
Our reading: OpenAI is unlikely to release a mindblowing GPT model in 2024.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Presumably the operating idea here is that (a) $100B will equal AGI and (b) that customers will pay literally anything for AGI. I doubt either premise is correct.”
Our reading: A $100B LLM investment is unlikely to produce AGI.
No horizon
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Forecast
“Presumably the operating idea here is that (a) $100B will equal AGI and (b) that customers will pay literally anything for AGI. I doubt either premise is correct.”
Our reading: It is doubtful that customers would pay literally anything for AGI.
No horizon
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Conditional forecast
“If they are wrong, the net effect will be to delay AGI, because their project will siphon away talent from other approaches – just as it already has, but more so.”
Our reading: If the Stargate premises are wrong, the project will delay AGI by siphoning talent away from other approaches.
No horizon
Only if: if the premises ($100B equals AGI; customers pay anything) are wrong
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Rhetoric
“A $100B LLM is still an LLM.”
Our reading: A $100B LLM is still an LLM.
No horizon
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“By any reasonable account the worst investment in the history of AI has to be the over $100 billion that have been invested in “driverless” cars.”
Our reading: By any reasonable account, the over $100 billion invested in driverless cars has to be the worst investment in the history of AI.
No horizon
Qualifier: by any reasonable account
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Observation
“The generality of Waymo’s approach is very much still in question, and I am not sure anyone else is still seriously in the full autonomy game.”
Our reading: The generality of Waymo's approach (to conditions such as bad weather, poorly mapped roads and different driving patterns) is very much still in question, and Marcus is not sure anyone else is still seriously in the full autonomy game.
No horizon
Qualifier: I am not sure
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Forecast
“Driverless cars were crippled by the endless long tail of outliers; LLMs may well pattern similarly, working in many cases but often troubled by the unexpected, even when scaled.”
Our reading: LLMs, even when scaled, may well remain often troubled by unexpected cases, as driverless cars were by the long tail of outliers.
No horizon
Qualifier: even when scaled
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Forecast
“Reasoning and planning in unexpected scenarios are still likely to be flawed.”
Our reading: Even a $100B LLM will likely still have flawed reasoning and planning in unexpected scenarios.
No horizon
Qualifier: in unexpected scenarios
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Forecast
“On many tasks, the results might be only slightly better than a $10B model or a $1B model, but the operating costs presumably would be much larger.”
Our reading: On many tasks, a $100B LLM's results might be only slightly better than those of a $10B or $1B model, while its operating costs would presumably be much larger.
No horizon
Scope: on many tasks
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Forecast
“On most problems, customers probably would not be willing to pay the required premium.”
Our reading: On most problems, customers probably would not pay the premium required by a $100B LLM.
No horizon
Qualifier: on most problems
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Forecast
“But misery loves company, and the forlorn $100B may see company before the end of the decade.”
Our reading: Before the end of the decade, another $100B AI investment (Microsoft/OpenAI's Stargate) may join driverless cars as a poor investment.
No horizon
Source: The Second Worst $100B Investment in The History of AI?
Assessment: Can’t be checkedSaid
Forecast
“Or to take another example, poorly-written software, authored in part by GenAI, could spread malware leading to a massive security incident that dwarfs WannaCry; who will pay the bills to fix it? Society.”
Our reading: Poorly written software authored in part by GenAI could spread malware, leading to a massive security incident that dwarfs WannaCry, and society would pay the bills to fix it.
No horizon
Scope: authored in part by GenAI
Source: The race between positive and negative applications of Generative AI is on – and not looking pretty
Assessment: Can’t be checkedSaid
Rhetoric
“They get rich, we pay the price.”
Our reading: AI companies get rich while society pays the costs.
No horizon
Source: The race between positive and negative applications of Generative AI is on – and not looking pretty
Assessment: Can’t be checkedSaid
Conditional forecast
“If the 2024 election flips based on GenAI-created disinformation, there will be a huge cost to society, but it is unlikely that the GenAI companies will be held responsible in any serious way.”
Our reading: If GenAI disinformation flips the 2024 election, GenAI companies are unlikely to be held seriously responsible.
No horizon
Only if: if the 2024 election flips based on GenAI disinformation
Source: The race between positive and negative applications of Generative AI is on – and not looking pretty
Assessment: Can’t be checkedSaid
Forecast
“It is doubtful that existing laws are up to the task, particularly given how expensive their lawyers are likely to be.”
Our reading: Existing laws are unlikely to make Microsoft or OpenAI pay for wrong chatbot advice.
No horizon
Qualifier: for wrong chatbot advice
Source: The race between positive and negative applications of Generative AI is on – and not looking pretty
Assessment: Can’t be checkedSaid
Conditional forecast
“If lots of code gets written, fast, but that code is riddled with security problems, the net advantage on the positive side of the ledger may be less than anticipated.”
Our reading: If lots of code gets written fast but that code is riddled with security problems, the net advantage of AI on the positive side of the ledger may be less than anticipated.
No horizon
Only if: if lots of code gets written, fast, but is riddled with security problems
Source: The race between positive and negative applications of Generative AI is on – and not looking pretty
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Climate impact needs to be considered, especially as models continue to grow.”
Our reading: The climate impact of AI models should be considered, especially as they grow.
No horizon
Qualifier: especially as models continue to grow
Source: Happy 1st Birthday, GPT-4
Assessment: Can’t be checkedSaid
Observation
“We might be reaching a plateau in terms of sheer capability.”
Our reading: LLM capability may be reaching a plateau around GPT-4 level.
No horizon
Source: Happy 1st Birthday, GPT-4
Assessment: Can’t be checkedSaid
Observation
“Meanwhile, phrases in official filings like“Charitable” and “For the public benefit” don’t seem to have meant much at all.”
Our reading: Phrases in OpenAI's official filings such as 'Charitable' and 'For the public benefit' don't seem to have meant much at all.
No horizon
Qualifier: don't seem to have
Source: OpenAI’s Lies and Half-Truths
Assessment: Can’t be checkedSaid
Observation
“Aside from all that, Murati claimed she had no idea what specific data Sora was trained on; given her role as CTO that seems all but inconceivable.”
Our reading: Given her role as CTO, it seems all but inconceivable to Marcus that Mira Murati had no idea what specific data Sora was trained on.
No horizon
Qualifier: given her role as CTO
Source: OpenAI’s Lies and Half-Truths
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“As the company gears up to upload your private email, calendar, and documents, possibly for eventual resale to advertisers, that’s something worth considering.”
Our reading: As OpenAI gears up to upload users' private email, calendar, and documents, possibly for eventual resale to advertisers, its lack of full candor is something worth considering.
No horizon
Qualifier: as the company gears up to upload your private email, calendar, and documents
Source: OpenAI’s Lies and Half-Truths
Assessment: Can’t be checkedSaid
Forecast
“I warned then that other areas would see the same. It looks like science is next.”
Our reading: It looks like science is the next area to be overrun by AI-generated fake submissions, as Clarkesworld was and as Marcus had warned other areas would be.
No horizon
Qualifier: it looks like
Assessment: Can’t be checkedSaid
Forecast
“If science journals and science itself are overrun by LLM-generated garbage, as now seems imminent, my agnostic “I can’t quite tell yet if LLMs will ultimately be of net benefit to society” is going to switch to “shut it down if they can’t fix this problem.””
Our reading: It seems imminent that science journals and science itself will be overrun by LLM-generated garbage.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If science journals and science itself are overrun by LLM-generated garbage, as now seems imminent, my agnostic “I can’t quite tell yet if LLMs will ultimately be of net benefit to society” is going to switch to “shut it down if they can’t fix this problem.””
Our reading: If science is overrun by LLM-generated garbage and the makers cannot fix it, LLMs should be shut down.
No horizon
Qualifier: if science journals are overrun by LLM-generated garbage
Assessment: Can’t be checkedSaid
Forecast
“And GenAI might do a lot worse than merely tripling workloads; the total number of articles may radically spike, many of them dubious and a waste of reviewers’ time.”
Our reading: Generative AI may cause the number of submitted scientific articles to spike radically, more than tripling reviewer workloads, with many dubious papers.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In my opinion, every article with ChatGPT remnants should be considered suspect and perhaps retracted, because hallucinations may have filtered in, and both authors and reviewers) were asleep at the switch.”
Our reading: Every scientific article containing ChatGPT remnants should be treated as suspect and perhaps retracted, since hallucinations may have entered it.
No horizon
Qualifier: articles containing ChatGPT remnants
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Literally the only other solution I can envision is to force LLM manufacturers to pay a significant tax to subsidize the journal review system that they are about to destroy.”
Our reading: The only other solution Marcus can envision, besides shutting LLMs down if they can't fix the problem, is to force LLM manufacturers to pay a significant tax to subsidize the journal review system that he says they are about to destroy.
No horizon
Qualifier: the only other solution I can envision
Assessment: Can’t be checkedSaid
Observation
“I realize now that I may have vastly underestimated the current scope of the problem.”
Our reading: The scope of generative AI's degradation of scientific literature may be much larger than Marcus previously estimated.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“GenAI-created garbage could quickly overwhelm their reviewing processes, leading to a precipitous drop in reputation.”
Our reading: GenAI-created garbage submissions could quickly overwhelm journals' reviewing processes and cause a precipitous drop in their reputation.
No horizon
Source: The Year of the Big-Balled Rat
Assessment: Can’t be checkedSaid
Forecast
“I understand the utility in using LLMs for writing and illustration etc, particularly for nonnative speakers, but the long term consequences on science may be severe.”
Our reading: Although using LLMs for writing and illustration has utility, particularly for nonnative speakers, the long-term consequences of LLM use on science may be severe.
No horizon
Qualifier: long term
Source: The Year of the Big-Balled Rat
Assessment: Can’t be checkedSaid
Forecast
“And it may get a lot worse.”
Our reading: Generative AI's damage to science may get a lot worse.
No horizon
Source: The Year of the Big-Balled Rat
Assessment: Can’t be checkedSaid
Forecast
“As ever, LLM developers are unlikely to be held to account.”
Our reading: LLM developers are unlikely to be held accountable for GenAI damage to scientific publishing.
No horizon
Source: The Year of the Big-Balled Rat
Assessment: Can’t be checkedSaid
Observation
“I am guessing there may be other errors in that paper, too, given the well-known tendency of LLMs to hallucinate in plausible ways that may initially escape notice.”
Our reading: LLMs tend to hallucinate in plausible ways that may escape notice, so the lithium battery paper may contain other errors.
No horizon
Source: The Year of the Big-Balled Rat
Assessment: Can’t be checkedSaid
Forecast
“Side note:, this is also a potential five-alarm fire for journal publishers, since for them reputation is all they have got.”
Our reading: GenAI-generated content in papers is a potential major crisis for journal publishers because their value rests on reputation.
No horizon
Source: The Year of the Big-Balled Rat
Assessment: Can’t be checkedSaid
Observation
“With the possible exception of the quick to rise and quick to fall alleged room-temperature superconductor LK-99, few things I have ever seen have been more hyped than generative AI.”
Our reading: Generative AI is among the most hyped things the author has ever seen, with the possible exception of LK-99.
No horizon
Qualifier: possible exception of LK-99
Assessment: Can’t be checkedSaid
Observation
““We are still a long way from machines that can genuinely understand human language”. Still true, though some have argued there is some superficial understanding.”
Our reading: As of March 2024, machines are still a long way from genuinely understanding human language.
No horizon
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But I still think we need a paradigm shift.”
Our reading: AI needs a paradigm shift beyond current deep learning to reach general intelligence.
No horizon
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Forecast
“Implict through the essay: scaling might not solve the problems not above. Pending/Still true”
Our reading: Scaling might not solve the fundamental problems of deep learning.
No horizon
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Conditional forecast
“We should never forget that the human brain is perhaps the most complicated system in the known universe; if we are to build something roughly its equal, open-hearted collaboration will be key.”
Our reading: If we are to build something roughly equal to the human brain, which is perhaps the most complicated system in the known universe, open-hearted collaboration will be key.
No horizon
Only if: if we are to build something roughly equal to the human brain
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Observation
“Likely still a concern, especially with open-source bots that may have less carefully constructed guardrails; there has been at least one chatbot related death.”
Our reading: Harmful chatbot exchanges with vulnerable users are likely still a concern, especially with open-source bots, and at least one death has been linked to a chatbot.
No horizon
Qualifier: especially open-source bots with weaker guardrails
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Observation
“Scaling the measures Kaplan and his OpenAI colleagues looked at—about predicting words in a sentence—is not tantamount to the kind of deep comprehension true AI would require” Still true, and increasingly acknowledged.”
Our reading: Getting better at next-word prediction through scaling does not amount to the deep comprehension true AI needs, and this is increasingly acknowledged.
No horizon
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Rhetoric
“Gary Marcus still feels like the field is mostly chasing ladders to the moon.”
Our reading: Marcus still feels the AI field is mostly chasing ladders to the moon.
No horizon
Scope: mostly
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Rhetoric
“None of this means we won’t eventually have new innovations, of some sort or another. Or that AGI is impossible.”
Our reading: None of the points in the post mean that new innovations of some sort will not eventually arrive, or that AGI is impossible.
No horizon
Qualifier: of some sort or another
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Observation
“We are still far from trustworthy AI.”
Our reading: As of March 2024, AI is still far from trustworthy.
No horizon
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Observation
“Neurosymbolic might be a promising alternative. Pending/still true”
Our reading: Neurosymbolic AI might be a promising alternative to pure deep learning.
No horizon
Source: Two years later, deep learning is still faced with the same fundamental challenges
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But casting a blind eye to covert racism, apparent copyright infringement, and violent and degrading outputs is not.”
Our reading: OpenAI should not turn a blind eye to covert racism, apparent copyright infringement, and violent or degrading outputs.
No horizon
Source: OpenAI’s “Own Goal”
Assessment: Can’t be checkedSaid
Rhetoric
“Leopards rarely change their spots.”
Our reading: Marcus applies the proverb 'leopards rarely change their spots' to OpenAI, expressing distrust of its character.
No horizon
Source: OpenAI’s “Own Goal”
Assessment: Can’t be checkedSaid
Observation
“The two facts that have come out make it look like they never really intended to follow their own charter.”
Our reading: The revealed facts suggest OpenAI never really intended to follow its charter.
No horizon
Source: OpenAI’s “Own Goal”
Assessment: Can’t be checkedSaid
Forecast
“Elon Musk’s lawsuit against OpenAI may fail, or even get dismissed.”
Our reading: Elon Musk's lawsuit against OpenAI may fail or even be dismissed.
No horizon
Source: OpenAI’s “Own Goal”
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Counterparts in other nations should consider doing the same.”
Our reading: Regulators outside the US should consider investigating LLM makers and demanding user logs.
No horizon
Source: Covert racism in LLMs
Assessment: Can’t be checkedSaid
Causal claim
“LLMs are, as I have been trying to tell you, too stupid to understand concepts like people and race; their fealty to superficial statistics drives this horrific stereotyping .”
Our reading: LLMs cannot understand concepts like people and race, and their reliance on superficial statistics causes racist stereotyping.
No horizon
Source: Covert racism in LLMs
Assessment: Can’t be checkedSaid
Observation
“What Hofman and his collaborators (including the MacArthur Fellow Dan Jurafsky) have documented may already be having real-world impact.”
Our reading: The covert racism documented in LLMs may already be having real-world impact.
No horizon
Source: Covert racism in LLMs
Assessment: Can’t be checkedSaid
Observation
“And Google’s debacle shows that guardrails are never easy..”
Our reading: Google's recent debacle shows that building guardrails for LLMs is never easy.
No horizon
Source: Covert racism in LLMs
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I would encourage Congress, the EEOC, HUD, the FTC, and others to investigate, and demand user logs and interaction data from the major LLM manufacturers.”
Our reading: Congress, the EEOC, HUD, the FTC and others should investigate covert racism in LLMs and demand user logs and interaction data from the major LLM makers.
No horizon
Source: Covert racism in LLMs
Assessment: Can’t be checkedSaid
Forecast
“The potential consequences on the world, given the widespread adoption of LLMs, are massive:”
Our reading: Because LLMs are so widely adopted, the potential consequences of their covert racism for the world are massive.
No horizon
Qualifier: given the widespread adoption of LLMs
Source: Covert racism in LLMs
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But the LLM companies should recall their systems until they can find an adequate solution.”
Our reading: LLM companies should recall their systems until they find an adequate solution to covert racism.
No horizon
Qualifier: until an adequate solution to covert racism is found
Source: Covert racism in LLMs
Assessment: Can’t be checkedSaid
Observation
“In my view, humanity has now taken a back seat to money; their choices now seem very directly anchored in a need to generate a massive return, in direct contradiction to their promises to California, to Musk, and to the world.”
Our reading: In Marcus's view, at OpenAI humanity has now taken a back seat to money, and OpenAI's choices now seem very directly anchored in a need to generate a massive return, in direct contradiction to its promises to California, to Musk, and to the world.
No horizon
Qualifier: in my view
Source: The Irony and the Agony of Elon Musk's Lawsuit against OpenAI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I honestly think the world would be a better place if it did.”
Our reading: The world would be better if OpenAI returned to its original nonprofit, open, humanity-benefiting mission.
No horizon
Qualifier: if OpenAI returned to what it was
Source: The Irony and the Agony of Elon Musk's Lawsuit against OpenAI
Assessment: Can’t be checkedSaid
Forecast
“We all want courts to administer justice, but in the end laws may or may not be enough to get OpenAI to honor its initial promises.”
Our reading: In the end, laws may or may not be enough to get OpenAI to honor its initial promises.
No horizon
Qualifier: in the end
Source: The Irony and the Agony of Elon Musk's Lawsuit against OpenAI
Assessment: Can’t be checkedSaid
Conditional forecast
“Humanity stands to lose if the lawsuit fails.”
Our reading: If Musk's lawsuit against OpenAI fails, humanity stands to lose.
No horizon
Only if: if Musk's lawsuit against OpenAI fails
Source: The Irony and the Agony of Elon Musk's Lawsuit against OpenAI
Assessment: Can’t be checkedSaid
Forecast
“Courtroom quarrels are likely to focus on documentation of their agreement, rather than the real issue: which is whether OpenAI is still built around helping all of humanity.”
Our reading: Courtroom disputes in Musk's lawsuit against OpenAI will likely focus on documentation of the agreement rather than whether OpenAI still serves all of humanity.
No horizon
Source: The Irony and the Agony of Elon Musk's Lawsuit against OpenAI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“And anyone making deals with Altman would be wise to read it.”
Our reading: Anyone making deals with Sam Altman should read Musk's lawsuit.
No horizon
Qualifier: anyone making deals with Altman
Source: Elon Musk sues OpenAI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I won’t predict who will win, but I will say that Elon has a point.”
Our reading: Elon Musk has a point in his lawsuit against OpenAI, Altman and Brockman, though Marcus declines to predict who will win.
No horizon
Qualifier: declines to predict who will win the lawsuit
Source: Elon Musk sues OpenAI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Yet another reason why we should never trust today’s chatbots.”
Our reading: People should never trust today's chatbots.
No horizon
Qualifier: today's chatbots
Assessment: Can’t be checkedSaid
Observation
“Things just got worse.”
Our reading: OpenAI's challenges got worse over the two weeks before late February 2024.
No horizon
Qualifier: in the last two weeks
Source: The last two weeks at OpenAI
Assessment: Can’t be checkedSaid
Rhetoric
“Waiting on RAG to fix what ails LLM is just waiting on another miracle.”
Our reading: Expecting RAG to fix LLMs' problems is like waiting for a miracle.
No horizon
Source: No, RAG is probably not going to rescue the current situation
Assessment: Can’t be checkedSaid
Conditional forecast
“Some great-grandchild of RAG, perhaps extended, revised, and elaborated, might even help — if people seriously embrace neurosymbolic AI in coming years.”
Our reading: Some great-grandchild of RAG, perhaps extended, revised, and elaborated, might help AI work reliably if people seriously embrace neurosymbolic AI in coming years.
No horizon
Only if: if people seriously embrace neurosymbolic AI in coming years
Source: No, RAG is probably not going to rescue the current situation
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Honestly, every LLM, with RAG or without, probably ought to have on of those old movie disclaimers after every response:”
Our reading: Every LLM, with or without RAG, should probably carry a disclaimer after each response.
No horizon
Qualifier: with RAG or without
Source: No, RAG is probably not going to rescue the current situation
Assessment: Can’t be checkedSaid
Conditional forecast
“Making it work is very likely going to require a whole new technology, one that is unlikely to arrive until the tech industry gets over its infatuation with LLMs and quick fixes.”
Our reading: Reliable AI will very likely require a whole new technology, which is unlikely to arrive until the tech industry moves past its infatuation with LLMs and quick fixes.
No horizon
Only if: new technology unlikely until industry gets over LLM infatuation and quick fixes
Source: No, RAG is probably not going to rescue the current situation
Assessment: Can’t be checkedSaid
Forecast
“Today’s issue may well be fixed quickly, but I hope it will be seen as the wakeup call that it is.”
Our reading: The ChatGPT malfunction of February 2024 may well be fixed quickly.
No horizon
Source: ChatGPT has gone berserk
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Today’s issue may well be fixed quickly, but I hope it will be seen as the wakeup call that it is.”
Our reading: The ChatGPT malfunction is a genuine wakeup call about the instability of generative AI and should be treated as one.
No horizon
Source: ChatGPT has gone berserk
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But I will quote something I said two weeks ago: “Please, developers and military personnel, don’t let your chatbots grow up to generals.””
Our reading: Developers and military personnel should not let their chatbots grow up to be generals.
No horizon
Source: ChatGPT has gone berserk
Assessment: Can’t be checkedSaid
Rhetoric
“We are still living in the age of machine learning alchemy that xkcd captured so well in a cartoon several years ago:”
Our reading: We are still living in the age of machine learning alchemy that an xkcd cartoon captured several years ago.
No horizon
Qualifier: still
Source: ChatGPT has gone berserk
Assessment: Can’t be checkedSaid
Rhetoric
“In the end, Generative AI is a kind of alchemy.”
Our reading: In the end, generative AI is a kind of alchemy.
No horizon
Qualifier: in the end
Source: ChatGPT has gone berserk
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The need for altogether different technologies that are less opaque, more interpretable, more maintanable, and more debuggable — and hence more tractable—remains paramount.”
Our reading: It is of paramount importance to develop altogether different AI technologies that are less opaque, more interpretable, maintainable, and debuggable than generative AI.
No horizon
Source: ChatGPT has gone berserk
Assessment: Can’t be checkedSaid
Rhetoric
“Gary Marcus keeps hoping that he will live to see an AI that is deeper than mere stochastic mimicry plus Band-Aids. He has, however, learned not to hold his breath.”
Our reading: Repeating a 2019 passage he wrote with Ernest Davis, Marcus says he keeps hoping to live to see an AI deeper than mere stochastic mimicry plus Band-Aids but has learned not to hold his breath.
No horizon
Qualifier: quoted from Rebooting AI (2019), co-written with Ernest Davis, and reaffirmed
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails
Assessment: Can’t be checkedSaid
Observation
“Some things have scaled exponentially, and some haven’t. The capacity to endow machines with the commonsense understanding of history, culture, and human values that would be required to make sensible guardrails has not.”
Our reading: Some things have scaled exponentially, but the capacity to give machines the commonsense understanding of history, culture, and human values needed to make sensible guardrails has not scaled exponentially.
No horizon
Qualifier: exponentially
Source: Has Google gone too woke? Why even the biggest models still struggle with guardrails