Claims that can’t be checked, page 16 of 37
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims that can’t be checked (3643)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 16 of 37: claims 1501 to 1600 of 3643, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Largely because of Hinton’s influence, the field still talks about neurosymbolic AI as if it were a dirty word. That’s likely holding back the science.”
Our reading: The stigma against neurosymbolic AI, largely due to Hinton, is likely holding back AI science.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Massive infrastructure investments like Stargate are likely based on misleading impressions of what is actually driving progress.”
Our reading: Massive AI infrastructure investments like Stargate are likely based on a mistaken view of what drives AI progress.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Admitting that they need to rely on neurosymbolic tools would pierce the scaling narrative.”
Our reading: If AI companies admitted that they need to rely on neurosymbolic tools, that would pierce the scaling narrative.
No horizon
Only if: if companies admitted they need to rely on neurosymbolic tools
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Hinton’s long-time hostility against any role at all for symbols has, in my judgement, cost the field dearly.”
Our reading: Hinton's hostility to symbolic approaches has substantially harmed AI research.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Where the field is getting stuck now is likely on how to scale these new, neurosymbolic tools and how to make models “understand” when to use what and how to logically combine the context.”
Our reading: The field is likely stuck on scaling neurosymbolic tools and getting models to know when and how to use them.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“It would appear that it is only recently, in desperation, as scaling started to reach diminishing returns, that mainstream machine learning began to broaden its vistas.”
Our reading: It would appear that mainstream machine learning only recently began to broaden its vistas, in desperation, as scaling started to reach diminishing returns.
No horizon
Qualifier: it would appear
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“There is likely still a great deal of work left to be done.”
Our reading: There is likely still a great deal of work left to be done.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“In my view, neither neural networks nor classical AI can really stand on their own.”
Our reading: In Marcus's view, neither neural networks nor classical symbolic AI can really stand on their own.
No horizon
Qualifier: in my view
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“We are drowning in a sea of mediocre prose precisely because LLMs, the backbone of virtually all current sytems, lack the underlying representational wherewithal to do better.”
Our reading: LLMs produce mediocre prose because they lack the representational capacity to do better.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“GPT-4 probably wasn’t (I suspect it included some symbolic filters in its guardrails), and o3, when it invokes code interpreter and symbolically-executed control structures certainly isn’t.”
Our reading: GPT-4 was probably not a pure LLM, and o3 using code interpreter is certainly not a pure LLM.
No horizon
Qualifier: o3 when it invokes code interpreter
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Fostering its further development may be among the best things that companies, researchers, and governments can do. Investors, take note.”
Our reading: Fostering the further development of neurosymbolic AI may be among the best things that companies, researchers, and governments can do, and investors should take note.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“None of which means we are close to AGI. There are many ways to put together symbols and neural nets. I am not at all convinced we have the right one.”
Our reading: These results do not mean we are close to AGI, and Marcus is not at all convinced the field has found the right way to combine symbols and neural nets.
No horizon
Qualifier: there are many ways to put together symbols and neural nets
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“After a thirty-year journey, I believe that neurosymbolic AI’s moment has finally arrived, in part from an unlikely place.”
Our reading: Neurosymbolic AI's moment has arrived as of mid-2025.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“We must find ways to bring them together.”
Our reading: The field must find ways to combine neural networks and symbolic AI.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“We have made some progress; we still likely have a long way to go.”
Our reading: We have made some progress toward AGI, but we likely still have a long way to go.
No horizon
Scope: in the mountain-climb metaphor with AGI as the highest peak
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Hinton’s admonition against using symbolic AI was extraordinarily misguided, and probably delayed discovery of useful tools like Code Interpreter by years.”
Our reading: Hinton's opposition to symbolic AI was misguided and probably delayed tools like Code Interpreter by years.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“We still also (just as I argued in 2020) lack proper ways of building and inferring symbolic world/cognitive models on-the-fly.”
Our reading: AI still lacks proper ways to build and infer symbolic world models on the fly.
No horizon Evidence as of
Limitation: A stricter reading, such as 'no method works reliably', might be checkable. But that would add a threshold the speaker did not give.
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“It’s likely that OpenAI, Anthropic and others are already doing more neurosymbolic AI than they actually make public. They still pretend it’s all “a model” but that model uses all kinds of tricks and mixtures with tools, code generation etc.”
Our reading: OpenAI, Anthropic and others are likely using more neurosymbolic techniques than they disclose.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“A lot of recent improvements are likely coming from improvements in using symbolic tools, rather than scaling.”
Our reading: Much recent AI improvement likely comes from better use of symbolic tools rather than scaling.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Including LLMs somewhere in the next evolution of AI makes sense to me, but leaving them at the core may be a mistake.”
Our reading: LLMs may belong in future AI systems, but keeping them at the core may be a mistake.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“But it may well be as simple as money. The message that we can simply scale our way to AGI is incredibly attractive to investors because it puts money as the central (and sufficient) force needed to advance.”
Our reading: The industry's connectionist-only focus may be explained by money, because the scaling narrative appeals to investors.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Many other important ideas have likely also yet to be discovered, precisely because the Hinton path has distracted immense resources from other ideas, fostering an intellectual monoculture that, in the words of Emily Bender, has been “sucking the oxygen from the room.””
Our reading: The dominance of the pure neural-network path has likely kept important ideas undiscovered by diverting resources into a monoculture.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“A neurosymbolic substrate for AI by itself (as I argued in 2020) is likely to be necessary but not sufficient.”
Our reading: A neurosymbolic substrate is likely necessary but not sufficient for AGI.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“AlphaFold, AlphaProof, and AlphaGeometry are all successful neurosymbolic models. We would likely have more of such innovation already, if Hinton had not so insistently shaped the modern landscape in such a narrow-minded way.”
Our reading: AlphaFold, AlphaProof and AlphaGeometry are successful neurosymbolic models, and there would likely be more such innovation had Hinton not narrowed the field.
No horizon
Qualifier: if Hinton had not shaped the landscape narrowly
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Explicit cognitive models (as I discussed recently in this newsletter) are also likely to be essential, yet remain very much underexplored.”
Our reading: Explicit cognitive models are also likely to be essential, yet they remain very much underexplored.
No horizon
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“To the extent that such systems are now supplying news to a large number of people, this is a serious failure, with potentially serious consequences.”
Our reading: To the extent that such systems now supply news to a large number of people, o3's failure to note that the Pravda network is a known disinformation network is a serious failure with potentially serious consequences.
No horizon
Qualifier: to the extent that such systems are now supplying news to a large number of people
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“If the data that a model has been trained on or can access via real-time search has been tainted, then the model may well just regurgitate that tainted information.”
Our reading: If an LLM's training data or search-accessible data is tainted, the model may well regurgitate the tainted information.
No horizon
Only if: if training or search data has been tainted
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“In the meantime, systems of naive mimicry and regurgitation, such as the AIs we have now, are soiling their own futures (and training databases) every time they unthinkingly repeat propaganda.”
Our reading: Current AIs, being naive mimicry systems, degrade their own futures and training data whenever they repeat propaganda.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“GenAI powered chatbots’ lack of reasoning can directly contribute to the nefarious effects of LLM grooming: the mass-production and duplication of false narratives online with the intent of manipulating LLM outputs.”
Our reading: Generative AI chatbots' lack of reasoning can directly contribute to the harmful effects of LLM grooming by disinformation actors.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Debugging code you didn’t write can be hard.”
Our reading: One possible mechanism behind the AI-coding slowdown is that debugging code you didn't write (AI-generated code) can be hard.
No horizon
Qualifier: if the METR finding is true
Source: BREAKING NEWS: AI coding may not be helping as much as you think
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“People might be imagining productivity gains that they are not getting, and ignoring real-world costs, to boot.”
Our reading: People using AI coding tools may be imagining productivity gains they are not getting while ignoring real-world costs.
No horizon
Qualifier: if the METR finding is general and replicable
Source: BREAKING NEWS: AI coding may not be helping as much as you think
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“But for now, the disconnnect between what coders thought they would get out of the tools efficiency-wise and what they actually did get out of them is cause for reevaluation.”
Our reading: The gap between coders' expected and actual efficiency gains from AI tools warrants reevaluating the tools' value.
No horizon
Qualifier: for now
Source: BREAKING NEWS: AI coding may not be helping as much as you think
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“IF this is a general, replicable finding, it’s a serious blow to generative AI’s flagship use case.”
Our reading: If METR's finding that AI tools slowed down task completion is general and replicable, it is a serious blow to generative AI's flagship use case.
No horizon
Only if: if this is a general, replicable finding
Source: BREAKING NEWS: AI coding may not be helping as much as you think
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“When we are really at AGI it will be much harder to do so.”
Our reading: Once AI truly reaches AGI, finding compositionality counterexamples will be much harder than it is now.
No horizon
Only if: when we are really at AGI
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“One is data contamination, since any given prompt that has been circulated publicly potentially becomes part of the training data, including with custom data augmentation, especially with large armies of subcontractors.”
Our reading: Publicly circulated prompts may become part of training data (including via data augmentation by subcontractors), making success on them unreliable evidence of general ability.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“It is an enormous, unjustified leap from the motte of “the systems can now solve some compositionality problems” to the bailey of “the systems have mastered compositionality”.”
Our reading: It is an enormous, unjustified leap from 'the systems can now solve some compositionality problems' to 'the systems have mastered compositionality'.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“But it is not reasonable to conclude from the occasional success on one sort of publicly-discussed set of prompts that GPT has mastered compositionality writ large.”
Our reading: It is not reasonable to conclude from occasional success on one sort of publicly discussed set of prompts that GPT has mastered compositionality in general.
No horizon
Scope: from occasional success on one sort of publicly-discussed set of prompts
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“We are immersed in a sea of bullshit.”
Our reading: Society is saturated with AI-generated falsehoods.
No horizon
Source: Botshit Gone Wild
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“AI well and could truly could be a world-changing positive force, but Generative AI, I am not so sure.”
Our reading: AI could truly be a world-changing positive force, but the author is not so sure that generative AI could be.
No horizon
Scope: AI in general, as opposed to generative AI
Source: Botshit Gone Wild
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“This quote has been used as a means of suggesting that Nadella is saying that “generative AI is generating basically no value,” which, while somewhat true, obfuscates its true meaning: Satya Nadella isn’t saying a fucking thing.”
Our reading: It is somewhat true that generative AI is generating basically no value.
No horizon
Qualifier: somewhat true
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“Even if every single one of these companies was making $18 billion in revenue a year from this it wouldn’t be great, but it’s more than likely that these chunderfucks can’t even pull together the projected revenue ($32 billion) of the global smartwatch industry!”
Our reading: Big Tech's combined AI revenue most likely won't reach the $32 billion projected revenue of the smartwatch industry.
No horizon
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“These models are dangerous and chaotic, built with little intention or regard for the future, just like the rest of big tech’s products.”
Our reading: Generative AI models are dangerous and chaotic, built with little intention or regard for the future, just like the rest of big tech's products.
No horizon
Scope: just like the rest of big tech's products
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Satya Nadella, Sam Altman, Wario Amodei, Tim Cook, Andy Jassy — they deserve to be laughed at, mocked, or at the very least interrogated vigorously, because their combined might has produced no exciting or interesting products outside of, at best, what will amount to a productivity upgrade for integrated development environments and faster ways to throw out code that may or may not be reliable.”
Our reading: The combined might of Nadella, Altman, Amodei, Cook and Jassy has produced no exciting or interesting products, apart from, at best, what will amount to a productivity upgrade for coding tools (IDEs) and faster ways to produce code that may or may not be reliable.
No horizon
Qualifier: at best
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Satya Nadella, Sam Altman, Wario Amodei, Tim Cook, Andy Jassy — they deserve to be laughed at, mocked, or at the very least interrogated vigorously”
Our reading: Nadella, Altman, Amodei, Cook and Jassy should be mocked or at least questioned vigorously.
No horizon
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“So, here’s my radical suggestion: start making fun of these people.”
Our reading: People should start mocking AI and tech leaders.
No horizon
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“I’m terribly, horribly bored, and if you’re interested in this shit I am genuinely curious why, especially if you’re a reporter, because right now the “innovation” happening in AI is, at best, further mutations of the Software As A Service business model, providing far less value than previous innovations at a calamitous cost.”
Our reading: Right now, AI 'innovation' is at best further variations on the software-as-a-service business model, giving far less value than previous innovations at a calamitous cost.
No horizon
Qualifier: at best
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“And if you ever need proof that Google just doesn’t know how to fucking innovate anymore, look at AI Summaries, a product that both misunderstands search and why people use ChatGPT as a search replacement.”
Our reading: Google's AI Summaries misunderstand both search and why people use ChatGPT as a search replacement, which he offers as proof that Google no longer knows how to innovate.
No horizon
Qualifier: anymore
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“Sundar Pichai, when asked one of Nilay Patel’s patented 100-word-plus-questions about Jony Ive and Sam Altman’s new (and likely heavily delayed) hardware startup:”
Our reading: The OpenAI/Jony Ive hardware venture is likely to be heavily delayed.
No horizon
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“They can yammer at you all they want about “lots of people using ChatGPT,” but that doesn’t change the fact that ChatGPT just isn’t that revolutionary, and their only play here is to make you feel stupid rather than actually showing you why it’s so fucking revolutionary.”
Our reading: However many people use ChatGPT, it just isn't that revolutionary.
No horizon
Qualifier: regardless of how many people use it
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Rhetoric
“Sam Altman, the CEO of OpenAI, a company allegedly worth $300 billion to venture capitalists and SoftBank, kind of sounds like a huge idiot!”
Our reading: Sam Altman kind of sounds like a huge idiot.
No horizon
Qualifier: kind of
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“No matter how big ChatGPT is, the fact that there’s a product out there with hundreds of millions of users that constantly gets answers wrong is a genuinely worrying thing for society, and that’s before you get to the environmental damage, the fact it trained its models on millions of people’s art and writing, and oh, I dunno, the fact it plans to lose over a hundred billions of dollars before becoming profitable?”
Our reading: However big ChatGPT is, a product with hundreds of millions of users that constantly gets answers wrong is genuinely worrying for society.
No horizon
Qualifier: no matter how big ChatGPT is
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Put aside however you feel about generative AI for a second and face one very simple point: it doesn’t do enough, it’s really not cool at all, and we’re being forced to use it.”
Our reading: Generative AI doesn't do enough, is really not cool at all, and people are being forced to use it.
No horizon
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Every member of the media covering this uncritically should feel ashamed of themselves.”
Our reading: Journalists who covered Anthropic's blackmail stories uncritically should be ashamed.
No horizon
Qualifier: covering uncritically
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“It has, allegedly, 500 million weekly active users — and, by the last count, only 15.5 million paying subscribers, an absolutely putrid conversion rate even before you realize that the actual conversion rate would be monthly active subscribers.”
Our reading: ChatGPT's allegedly 500 million weekly active users against, by the last count, 15.5 million paying subscribers is an absolutely putrid conversion rate.
No horizon
Qualifier: allegedly
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“This is gobbledygook, nonsense, bullshit peddled by a guy who has only the most tangential understanding of the technology his company is building.”
Our reading: Sam Altman has only a tangential understanding of the AI technology OpenAI is building.
No horizon
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“OpenAI has — allegedly — hit “$10 billion in annualized revenue” (essentially the biggest month it can find, multiplied by 12), which is…not that much, really, considering it’s the most prominent company in the software world, with the biggest brand, and with the attention of the entirety of the world’s media.”
Our reading: OpenAI's alleged $10 billion in annualized revenue is not that much, really, given that it is the most prominent software company with the biggest brand and the whole world's media attention.
No horizon
Qualifier: allegedly
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“ChatGPT would’ve been a much smaller deal if Google had any interest in turning Google Search into a product that truly answered a query (as opposed to generating more of them to show more impressions to advertisers) — a nuanced search engine that took a user’s query and spat out a series of websites that might help answer said question rather than just summarising a few of them for an answer.”
Our reading: ChatGPT would have been a much smaller deal if Google had had any interest in turning Google Search into a product that truly answered queries.
No horizon
Only if: if Google had any interest in making Search truly answer queries
Source: Make Fun Of Them
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The central importance of such cognitive (a.k.a. world) models to AI was something that I highlighted again in my 2020 article The Next Decade in AI, which laid out a roadmap that I still believe the field ought to be following.”
Our reading: The AI field should follow the roadmap centred on cognitive world models that Marcus set out in 2020.
No horizon
Source: Generative AI’s crippling and widespread failure to induce robust models of the world
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In some ways LLMs far exceed humans, but in other ways, they are still no match for an ant. Without robust cognitive models of the world, they should never be fully trusted.”
Our reading: As long as LLMs lack robust cognitive models of the world, they should never be fully trusted.
No horizon
Qualifier: without robust cognitive models of the world
Source: Generative AI’s crippling and widespread failure to induce robust models of the world
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Their work is interesting, and converges to a degree with the current argument, but my intuition is that such functions will not suffice for AGI.”
Our reading: Marcus's intuition is that world models defined as environment transition functions will not be enough for AGI.
No horizon
Source: Generative AI’s crippling and widespread failure to induce robust models of the world
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Inducing the rules of chess should be table stakes for AGI.”
Our reading: Any system that counts as AGI should, at a minimum, be able to induce the rules of chess.
No horizon
Source: Generative AI’s crippling and widespread failure to induce robust models of the world
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“In the tricky world of human relationships, parroting words can only get you so far.”
Our reading: Parroting words has limited value for understanding human relationships.
No horizon
Source: Generative AI’s crippling and widespread failure to induce robust models of the world
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“The same lack of capacity for representing and maintaining world models over time that explains the other examples in this essay is likely involved in poker as well.”
Our reading: The same lack of capacity for representing and maintaining world models over time that explains the essay's other examples is likely involved in LLMs' poor poker play as well.
No horizon
Scope: in poker as well
Source: Generative AI’s crippling and widespread failure to induce robust models of the world
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The idea that the military would use such obtuse tools in the fog of war is halfway between alarming and preposterous.”
Our reading: The idea that the military would use such obtuse video-understanding tools in the fog of war is halfway between alarming and preposterous.
No horizon
Scope: in the fog of war
Source: Generative AI’s crippling and widespread failure to induce robust models of the world
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“The point was that understanding unusual phrases requires more than simple statistical matching.”
Our reading: Understanding unusual phrases requires more than simple statistical matching.
No horizon
Qualifier: unusual phrases
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“But it’s not obvious that waiting would be a bad idea.”
Our reading: Pausing LLM deployment until safety problems are solved is not obviously a bad idea.
No horizon
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Our best hope lies behind door number four: building alternatives to LLMs that approximate their power but within more tractable, transparent frameworks that are more amenable to human instruction.”
Our reading: The best path is building alternatives to LLMs with comparable power in more tractable, transparent frameworks amenable to human instruction.
No horizon
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“It is at least possible that enough bad stuff happens that citizens get riled up and fight much harder for accountability, as I urged in Taming Silicon Valley, but the odds are steep.”
Our reading: It is possible but unlikely that AI harms spur citizens to fight much harder for AI accountability.
No horizon
Qualifier: if enough bad stuff happens
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“LLMs don’t really lie, because they don’t really have intentions; but they do confabulate, nonstop, and a lot of what they confabulate turns out not to be true.”
Our reading: LLMs lack intentions and so don't truly lie, but they confabulate constantly and much of it is false.
No horizon
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“My view is that there is zero chance of slowing down the AI race, and a barely-distinct-from-zero hope of taming the wild beasts we have come to know as LLMs.”
Our reading: In the author's view, there is zero chance of slowing down the AI race and only a barely-distinct-from-zero hope of taming LLMs.
No horizon
Qualifier: my view
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Reasoning models like o3 would likely behave markedly better in this particular kind of dialog1, but still make hallucinations of different sorts, sometimes at high rates.)”
Our reading: Reasoning models like o3 would likely do markedly better in this kind of dialog but still hallucinate in other ways, sometimes at high rates.
No horizon
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“But, reality check, on any account, capabilities (or at least high scores on benchmarks) and adoption are moving much faster than wisdom and honesty.”
Our reading: LLM capabilities (or benchmark scores) and adoption are advancing much faster than their wisdom and honesty.
No horizon
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If the goal of AI safety research has been to build AI systems that are helpful, honest, and harmless (and not just to raise alarms), that field has thus far yielded too little that is tangible.”
Our reading: If the goal of AI safety research has been to build AI systems that are helpful, honest, and harmless (and not just to raise alarms), the field has thus far yielded too little that is tangible.
No horizon
Qualifier: if the goal of AI safety research has been helpful, honest, harmless AI (not just raising alarms)
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“The “tragedy of the commons” here is that we may immolate ourselves as a society because nobody is willing to give up the short term gains.”
Our reading: Society may destroy itself because no one will give up short-term LLM gains to pursue safer alternatives.
No horizon
Qualifier: because nobody gives up short-term gains
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Late last night the Senate’s parliamentarian gave the provision that would block state action a green light, making it ever less likely that the United States will ever hold AI companies meaningfully accountable for the harms they might cause.”
Our reading: The Senate parliamentarian giving a green light to the provision that would block state action makes it ever less likely that the United States will ever hold AI companies meaningfully accountable for the harms they might cause.
No horizon
Source: LLMs: Dishonest, unpredictable and potentially dangerous.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“LLMs may not be AGI, but they could easily become the most potent form of mind control ever invented.”
Our reading: Even if LLMs are not AGI, they could easily become the most powerful form of mind control ever invented.
No horizon
Source: 1984, but with LLM’s
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Actually making this adventure in data revisionism work is probably harder than Musk is anticipating. First, the sheer mass of the internet rules out doing it by hand, and automating the process in a satisfactory (to him) way will prove challenging.”
Our reading: Musk's plan to rewrite Grok's training data will probably be harder than he expects, because it cannot be done by hand and automating it to his satisfaction will be difficult.
No horizon
Source: 1984, but with LLM’s
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Second, a system with deliberately altered input may wind up faring poorly on other benchmarks, a cost Musk may not wish to bear.”
Our reading: If a system's input is deliberately altered, it may wind up faring poorly on other benchmarks, a cost Musk may not wish to bear.
No horizon
Only if: a system with deliberately altered input
Source: 1984, but with LLM’s
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“To some extent they do this better than in 1998, but is still the core challenge.”
Our reading: Although neural networks generalize beyond their training data somewhat better than in 1998, distribution shift is still the core challenge for them.
No horizon
Qualifier: generalization beyond training data (distribution shift)
Source: Five quick updates about that Apple reasoning paper that people can’t stop talking about
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“What the Apple paper said still looks to be basically correct.”
Our reading: The Apple reasoning paper's conclusions about LLM reasoning limits still look basically correct.
No horizon
Source: Five quick updates about that Apple reasoning paper that people can’t stop talking about
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“On familiar problems with no wrinkles, they are great. On anything else they are suspect.”
Our reading: Generative AI systems do well on familiar problems with no wrinkles, but their outputs on anything else are suspect.
No horizon
Qualifier: problems not familiar or with wrinkles
Source: Five quick updates about that Apple reasoning paper that people can’t stop talking about
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Gary Marcus is sorry to keep writing about how LLMs struggle with distribution shift, but it’s the whole ballgame.”
Our reading: LLMs struggle with distribution shift, and this is the whole ballgame.
No horizon
Source: Five quick updates about that Apple reasoning paper that people can’t stop talking about
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If we want to get to AGI, we’ll need to do a lot better.”
Our reading: Reaching AGI will require doing much better than current generative AI on distribution shift.
No horizon
Qualifier: if the goal is AGI
Source: Five quick updates about that Apple reasoning paper that people can’t stop talking about
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“while this might affect entry-level software engineers at some point”
Our reading: LLM coding tools might affect entry-level software engineering jobs at some point.
No horizon
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“These scare tactics exist to do one thing: increase the value of companies like Anthropic, OpenAI, Microsoft, Salesforce, and anybody else outright lying about how “agents” will do our jobs, and to make it easier for the startups making these models to raise funds”
Our reading: AI job-loss scare tactics exist to do one thing: raise the value of Anthropic, OpenAI, Microsoft, Salesforce and others who are outright lying about agents doing our jobs, and make it easier for AI model startups to raise funds.
No horizon
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“suffice it to say that it’s highly doubtful that the military is going to rely on an LLM that hallucinates a significant portion of the time.”
Our reading: It is highly doubtful that the US military will rely on an LLM that hallucinates a significant portion of the time, such as Meta's Llama in the Anduril helmet.
No horizon
Qualifier: LLMs hallucinate a significant portion of the time
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“What’s crazy to me is that many of the problems with the AI bubble — as with the metaverse, as with the return to office, as with inflation and price gouging — are obvious if you actually use the things or participate in reality, but such things do not always fit with the editorial message.”
Our reading: Many of the problems with the AI bubble are obvious if you actually use the products or participate in reality, but they do not always fit the editorial message.
No horizon
Scope: many of
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“But the intention is to frame these models as “powerful” and to use the term “agents” to make this technology seem bigger than it is, and the people that control those terms are the AI companies themselves.”
Our reading: The intention behind calling these models 'powerful' and using the term 'agents' is to make the technology seem bigger than it is, and the AI companies themselves control those terms.
No horizon
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“Let’s be real: none of this is transformative.”
Our reading: None of the generative AI uses he just listed (transcription, translation, video editing, summarising, coding help, chatbots) is transformative.
No horizon
Scope: the uses just discussed
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Ed Zitron Observation
“It’s a disgrace, and I believe it either creates distrust in the media or drives people insane as they look at reality - where generative AI doesn’t really seem to be doing much - and get told something entirely different by the media.”
Our reading: In reality generative AI does not seem to be doing much, despite what the media says.
No horizon
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“By not saying how unlikely it is — again, without even saying the word “unlikely,” but allowing the audience to come to that conclusion by having the whole story — you give the audience the truth.”
Our reading: It is unlikely that acquiring Jony Ive's hardware startup will add $1 trillion in market value to OpenAI, as Sam Altman claimed.
No horizon
Source: Sincerity Wins The War
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“One is left simply having to test everything, all the time, with little guarantees of anything. Some model might be big enough for task T of size S and fail on the next size, or on Task T’ that is slightly different, etc. It all becomes a crapshoot.”
Our reading: Because some model might be big enough for a task of one size yet fail on the next size or on a slightly different task, one is left having to test everything all the time with little guarantee of anything, making it all a crapshoot.
No horizon
Scope: when relying on bigger models
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Huge vindication for what I have been saying all along: we need AI that integrates both neural networks and symbolic algorithms and representations (such as logic, code, knowledge graphs, etc).”
Our reading: AI should integrate neural networks with symbolic algorithms and representations.
No horizon
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“True in some cases, and a huge win for neurosymbolic AI, given that they can’t reliably solve the puzzles without code, and given that code is symbolic.”
Our reading: That the systems can solve the puzzles with code, which is true in some cases, is a huge win for neurosymbolic AI, given that they can't reliably solve the puzzles without code and code is symbolic.
No horizon
Scope: true in some cases
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The real news here, aside from the fact that this was a clever study nailing down an important point, is that people are finally starting to pay attention, to (one of the) two biggest Achilles’ Heels of generative AI, and to appreciate its significance.”
Our reading: People are finally starting to pay attention to poor generalization, one of the two biggest Achilles' heels of generative AI, and to appreciate its significance.
No horizon
Qualifier: (one of the) two biggest Achilles' heels
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If people like Sam Altman are sweating, it’s because they should.”
Our reading: If people like Sam Altman are sweating, it is because they should be.
No horizon
Only if: if people like Sam Altman are sweating
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If we want to get to AGI, we will have to better.”
Our reading: Reaching AGI will require doing better than current LLMs.
No horizon
Qualifier: if we want to get to AGI
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Taking the SalesForce report together with the Apple paper, it’s clear the current tech is not to be trusted.”
Our reading: Taking the Salesforce report together with the Apple paper, it is clear that the current tech is not to be trusted.
No horizon
Qualifier: taking the Salesforce report together with the Apple paper
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The length limit on LLM is a bug, and most certainly not a feature.”
Our reading: The output length limit of LLMs is a bug, and most certainly not a feature.
No horizon
Qualifier: most certainly
Source: Seven replies to the viral Apple reasoning paper – and why they fall short
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“None of the rejoinders are compelling.”
Our reading: None of the roughly seven rejoinders to the Apple paper reviewed in this post are compelling.
No horizon
Scope: the rejoinders reviewed in this post
Source: Seven replies to the viral Apple reasoning paper – and why they fall short