Unresolved claims, page 5 of 8
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Unresolved claims (795)
The record was checked, and the evidence available by the cutoff does not settle it. Forecasts whose deadline has not arrived are listed apart, as not yet due.
Page 5 of 8: claims 401 to 500 of 795, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Unfortunately the key test was not done.”
Our reading: The key test of o3's base model on ARC without pretraining was not done.
No horizon Evidence as of
Limitation: A narrower reading confined to ARC Prize's own evaluation would be easier to support.
Source: 𝗼𝟯 “𝗔𝗥𝗖 𝗔𝗚𝗜” 𝗽𝗼𝘀𝘁𝗺𝗼𝗿𝘁𝗲𝗺 𝗺𝗲𝗴𝗮𝘁𝗵𝗿𝗲𝗮𝗱: 𝘄𝗵𝘆 𝘁𝗵𝗶𝗻𝗴𝘀 𝗴𝗼𝘁 𝗵𝗲𝗮𝘁𝗲𝗱, 𝘄𝗵𝗮𝘁 𝘄𝗲𝗻𝘁 𝘄𝗿𝗼𝗻𝗴, 𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗮𝗹𝗹 𝗺𝗲𝗮𝗻𝘀
Assessment: UnresolvedSaid
Gary Marcus Forecast
“In 2025, a lot of AI influencers and maybe some companies are going to claim we have reached AGI.”
Our reading: During 2025, many AI influencers and possibly some companies will claim AGI has been reached.
Deadline passed Evidence as of
Limitation: The evidence is a selected set of statements, not a census.
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: UnresolvedSaid
Gary Marcus Observation
“Even setting aside the 1000x costs, not one person outside of OpenAI has evaluated o3’s robustness across different types of problems.”
Our reading: No one outside OpenAI has evaluated o3's robustness across different problem types.
No horizon Evidence as of
Limitation: A broader reading of problem types could count ARC tasks and produce a different assessment.
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: UnresolvedSaid
Gary Marcus Observation
“Most commentary has ignored one or both of these ideas.”
Our reading: Most commentary on o3 has ignored one or both of two ideas: that passing ARC may be necessary but not sufficient for AGI, and that a mechanism can be effective in some domains but not others.
No horizon Evidence as of
Limitation: The packet provides no representative sampling frame, denominator, systematic coding, or uncertainty estimate for commentary published between the December 20 announcement and Marcus’s December 21 post.
Source: o3, AGI, the art of the demo, and what you can expect in 2025
Assessment: UnresolvedSaid
Gary Marcus Observation
“OpenAI has yet to make a profit, and doesn’t anticipate making one before 2029.”
Our reading: OpenAI has not yet made a profit and does not expect to before 2029.
No horizon Evidence as of
Limitation: The original financial reporting could not be read with the required tool.
Source: Casey (Newton) Strikes Out
Assessment: UnresolvedSaid
Gary Marcus Observation
“Newton is also conveniently ignoring an absolutely fundamental distinction, between early period of reliable, rapid progress from 2019-2022 (when training for GPT-4) was completed, and a more recent regime in which by many measures, progress has (reportedly) slowed.”
Our reading: After rapid progress in 2019-2022, LLM progress has by many measures reportedly slowed.
No horizon Evidence as of
Limitation: A narrower assessment asking only whether slowdown reports existed would be supported.
Source: Casey (Newton) Strikes Out
Assessment: UnresolvedSaid
Gary Marcus Observation
“Diminishing returns doesn’t mean zero improvement, it means that the “laws” of scaling no longer hold, and that the ROI on adding more data and compute are no longer are predictive in the way that they used to be, hence the need for new approaches.”
Our reading: The diminishing returns he describes in AI do not mean zero improvement; they mean the scaling "laws" no longer hold and the returns on adding more data and compute are no longer predictive in the way they used to be, which is why new approaches are needed.
No horizon Evidence as of
Limitation: Important frontier training results were private.
Source: Casey (Newton) Strikes Out
Assessment: UnresolvedSaid
Gary Marcus Observation
“We still don’t have driverless cars in 97% of the world after a decade’s promises; hallucinations haven’t gone away. Home humanoid robots are nowhere in sight.”
Our reading: Driverless cars are absent from 97% of the world, hallucinations persist, and home humanoid robots are nowhere in sight.
No horizon Evidence as of
Limitation: The percentage is not reproducible without a denominator.
Source: Casey (Newton) Strikes Out
Assessment: UnresolvedSaid
Gary Marcus Forecast
“In February, the day after Sora came out, I warned that it didn’t really understand physics—and wouldn’t — in essay called Sora’s Surreal Physics:”
Our reading: Sora does not really understand physics and will not come to understand it.
No deadline stated Evidence as of
Limitation: Understanding has no speaker-specified test or threshold.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Sora is never really getting it, even with mountains and mountains of data.”
Our reading: Sora will never really grasp object permanence, even with vast amounts of data.
No deadline stated Evidence as of
Limitation: The cited aggregate sentence does not give a Sora-only success rate.
Assessment: UnresolvedSaid
Gary Marcus Observation
“The third illustrates that it cannot reliably eliminate an object part.”
Our reading: ChatGPT with DALL-E 3 cannot reliably eliminate a part of an object from an image.
No horizon Evidence as of
Limitation: The packet contains no independently verified November 2024 omission outputs or directly relevant contemporary evaluation.
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: UnresolvedSaid
Gary Marcus Observation
“But current technology has proven quite poor at such sanity checks. New technology may be needed.”
Our reading: Current AI technology is quite poor at checking whether its own image output satisfies the prompt and feedback, and new technology may be needed.
No horizon Evidence as of
Limitation: The evidence concerns related checking tasks, not an exact replication of the claimed workflow.
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: UnresolvedSaid
Gary Marcus Observation
“The first example illustrates that ChatGPT with DALL-E 3 is unable to reliably associate labels correctly with the parts of an object.”
Our reading: ChatGPT with DALL-E 3 cannot reliably associate labels correctly with the parts of an object.
No horizon Evidence as of
Limitation: Unresolved does not mean the bicycle example was correct.
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: UnresolvedSaid
Gary Marcus Observation
“In particular, DALL-E2 was very poor at following specifications about parts of objects.”
Our reading: As of October 2022, DALL-E2 was very poor at following prompt specifications about parts of objects.
No horizon Evidence as of
Limitation: This is an evidence gap, not evidence that DALL-E2 performed well.
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: UnresolvedSaid
Gary Marcus Observation
“We did not see anything like the kind of improvement that ChatGPT has exhibited over math word problems, over the last two years.”
Our reading: In the authors' body-parts image experiments, they did not see anything like the kind of improvement that ChatGPT has shown on math word problems over the last two years.
No horizon Evidence as of
Limitation: Related benchmark gains cannot substitute for the complete comparison.
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: UnresolvedSaid
Gary Marcus Observation
“The second example illustrates that it cannot reliably produce a non-standard number of body parts.”
Our reading: ChatGPT with DALL-E 3 cannot reliably produce images with a non-standard number of body parts.
No horizon Evidence as of
Limitation: No verified contemporary prompt-and-image evaluation directly measures requested nonstandard anatomical counts.
Source: Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
Assessment: UnresolvedSaid
Gary Marcus Observation
“This just in, right on the heels of Satya Nadella also publicly acknowledging diminishing returns:”
Our reading: Satya Nadella has publicly acknowledged diminishing returns.
No horizon Evidence as of
Limitation: Acknowledging possible limits could support a weaker interpretation of Marcus's wording.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Two years on, on the most important question of all – factuality and reliability — we are still pretty much where we were when ChatGPT first came out: wishing and hoping.”
Our reading: Two years after ChatGPT came out, on factuality and reliability, which he calls the most important question, we are still pretty much where we were at its release: wishing and hoping.
No horizon Evidence as of
Limitation: Neither study establishes aggregate progress from November 2022 to November 2024 across representative factuality and reliability tasks.
Source: ChatGPT, at Age Two
Assessment: UnresolvedSaid
Gary Marcus Observation
“There has been a ton of experimentation, but relatively few well-documented success stories, and some fairly negative stories like a recent one at Business Insider on GPT-powered CoPilot,”
Our reading: There has been much experimentation with ChatGPT-style AI but relatively few well-documented success stories.
No horizon Evidence as of
Limitation: The survey measures deployment and measurement practices, not the number of public success stories.
Source: ChatGPT, at Age Two
Assessment: UnresolvedSaid
Gary Marcus Observation
“But a lot of of theories about how ChatGPT would be used in practice have fizzled out.”
Our reading: Many theories about how ChatGPT would be used in practice have fizzled out.
No horizon Evidence as of
Limitation: The population of predictions is undefined.
Source: ChatGPT, at Age Two
Assessment: UnresolvedSaid
Gary Marcus Observation
“Yet hundreds of billions more have been invested on further speculation that scaling would somehow magically cure problems that actually appear to be inherent with the technology.”
Our reading: Hundreds of billions more have been invested on further speculation that scaling would somehow magically cure problems that appear to be inherent to the technology.
No horizon Evidence as of
Limitation: The spending categories cannot be added without checking overlap and accounting treatment.
Source: ChatGPT, at Age Two
Assessment: UnresolvedSaid
Gary Marcus Observation
“But things have been changing rapidly, and the great scaling retrenchment has already begun.”
Our reading: As of November 2024, a major retreat from scaling claims in the AI field has already begun.
No horizon Evidence as of
Limitation: Great retrenchment has no stated threshold.
Assessment: UnresolvedSaid
Gary Marcus Observation
“The new version is that there is not one scaling law, but three: scaling with how long you train a model (which isn’t really holding anymore), scaling with how long you post-train a model, and scaling with how long you let a given model wrestle with a given problem (or what Satya Nadella called scaling with “inference time compute”).”
Our reading: Scaling performance by training models longer is no longer really holding.
No horizon Evidence as of
Limitation: The strongest experimental counterevidence concerns an earlier model and is context only.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Even if you look at OpenAI’s own research post on o1, which introduced the newest putative law, it is immediately apparent that scaling test time compute works for some problems and not others.”
Our reading: OpenAI's own o1 research shows that scaling test-time compute helps on some problems but not others.
No horizon Evidence as of
Limitation: The conclusion depends on distinguishing model comparisons from compute-scaling experiments.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Like the first generalization, the new ones will eventually run out.”
Our reading: The new scaling laws (post-training and inference-time) will eventually stop holding.
No deadline stated Evidence as of
Limitation: A narrower reading limited to particular scaling recipes would be easier to support.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Hundreds of billions of dollars were invested on the assumption that I was wrong, largely to the exclusion of investment in other approaches.”
Our reading: Hundreds of billions of dollars went into LLM scaling on the assumption that scaling would keep working, and alternative approaches got little investment.
No horizon Evidence as of
Limitation: The capex figure is a projection made in November 2024 for the full year, not a realized total.
Source: Satya Nadella and the three stages of scientific truth
Assessment: UnresolvedSaid
Gary Marcus Observation
“Crucially, there has never been any principled argument that they would solve hallucinations or reasoning, or edge cases in open-ended worlds —or that synthetic data would suffice indefinitely to keep feeding the machine.”
Our reading: There has never been a principled argument that scaling would solve hallucinations, reasoning, or open-world edge cases, or that synthetic data would suffice indefinitely.
No horizon Evidence as of
Limitation: The claim concerns the existence of arguments, not whether those arguments ultimately proved correct.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Meanwhile, precious little investment has been made in other approaches.”
Our reading: Very little investment has gone into AI approaches other than LLMs.
No horizon Evidence as of
Limitation: The observation period is an explicit annual-reporting interpretation.
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Given the cost of chips, profits will be elusive.”
Our reading: Because chips are costly, LLM companies will struggle to make profits.
No deadline stated Evidence as of
Limitation: The cited financial evidence concerns one developer.
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Gary Marcus Observation
“Just as I argued here in April 2024, LLMs have reached a point of diminishing returns.”
Our reading: LLMs have reached a point of diminishing returns from scaling.
No horizon Evidence as of
Limitation: The evidence lacks a comparable series of frontier training runs measuring capability gains against proportional increases in compute and data [e1, e2].
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Gary Marcus Forecast
“LLMs such as they are, will become a commodity; price wars will keep revenue low.”
Our reading: LLMs such as they are will become a commodity, and price wars will keep revenue low.
No deadline stated Evidence as of
Limitation: The market study is a preprint.
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“If enthusiasm for GenAI dwindles and market valuations plummet, AI won’t disappear, and LLMs won’t disappear; they will still have their place as tools for statistical approximation.”
Our reading: If enthusiasm for GenAI dwindles and market valuations plummet, AI and LLMs will not disappear, and LLMs will still have their place as tools for statistical approximation.
No deadline stated Evidence as of
Only if: if enthusiasm for GenAI dwindles and market valuations plummet
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Gary Marcus Observation
“He then went on to extrapolate that it would be 10,000 times better in 4 years, never considering potential bottlenecks around data, or compute, nor inherent problems with LLMs.”
Our reading: When Musk extrapolated that AI would be 10,000 times better in 4 years, he did not consider potential bottlenecks around data or compute, or inherent problems with LLMs.
No horizon Evidence as of
Limitation: The only item is one news report. It covers parts of Musk's remarks, not the full session.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“(Side bonus: GenAI uses a massive amount of power, to the point of keeping fossil fuels in vogue, and that’s good for oil-rich countries, even if GenAI never becomes reliable.)”
Our reading: Generative AI uses so much power that it keeps fossil fuels in demand, which benefits oil-rich countries whether or not GenAI ever becomes reliable.
No horizon Evidence as of
Limitation: The evidence covers data-centre demand in general, not generative AI specifically.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“It’s just fantasy to imagine selling 200 times as many humanoid robots in the nearish term when nobody knows how to build a single safe, reliable, generally useful humanoid right now, at any price.””
Our reading: Humanoid robots will not sell 200 times Roomba's roughly 50 million units in the near term.
No deadline stated Evidence as of
Limitation: The threshold inherits the approximation in Marcus's Roomba baseline.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“The same will be true for humanoid robots, and we aren’t going to see six humanoid robots for every car anytime soon.”
Our reading: Many people will not be able to afford or want humanoid robots, and there will not be six humanoid robots per car anytime soon.
No deadline stated Evidence as of
Limitation: Willingness to pay is different from inability to afford a purchase.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Meanwhile GPT-5 still hasn’t arrived, and nor has the much ballyhooed Sora.”
Our reading: As of October 2024, neither GPT-5 nor Sora has been released.
No horizon Evidence as of
Limitation: The required event-date restriction prevents the later launch announcements from serving as decisive evidence.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Those products have not by a longshot lived up to everyone’s expectations, but they do actually ship, and customers can try those products out and decide for themselves. It’s hard to call that smoke and mirrors.”
Our reading: OpenAI's products have not by a longshot lived up to everyone's expectations, but they do actually ship and customers can try them, so it is hard to call OpenAI smoke and mirrors.
No horizon Evidence as of
Limitation: Scoring only product access would omit an explicit part of the listed proposition.
Assessment: UnresolvedSaid
Gary Marcus Observation
“But then again, their products have always been served with large side order of bullshit, like when Sam said, apparently jokingly, that “AGI has been achieved internally”. None of the robot stuff they promised in the Rubik’s cube campaign ever shipped.”
Our reading: OpenAI's products have always come with lots of hype, and none of the robotics promised in its Rubik's cube campaign ever shipped.
No horizon Evidence as of
Limitation: The distinction between research ambitions and product promises materially affects the assessment.
Assessment: UnresolvedSaid
Gary Marcus Observation
“But the anti-neurosymbolic tradition is limited; its most visible manifestation, LLMs, has brought fame and money, but no robust solution to solving any particular problem with great reliability.”
Our reading: The anti-neurosymbolic tradition is limited. Its most visible manifestation, LLMs, has brought fame and money but no robust solution to solving any particular problem with great reliability.
No horizon Evidence as of
Limitation: High accuracy is evidence of useful capability and should not be dismissed merely because errors remain.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Almost every putative virtue of AI (aside from making a small number of people rich) has been promissory.”
Our reading: Almost every claimed benefit of AI, other than enriching a few people, has so far been only a promise.
No horizon Evidence as of
Limitation: The search identified examples and a clinical review, not a representative survey of AI promises.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Even though some of the drawbacks, which inherently are less dependent on reliably and tight causal reasoning, such as misinformation use, nonconsensual deepfake porn, ans deepfake kidnapping scams, are already here to say).”
Our reading: Some of AI's drawbacks, which are inherently less dependent on reliable and tight causal reasoning, such as misinformation, nonconsensual deepfake porn, and deepfake kidnapping scams, are already here to stay.
No horizon Evidence as of
Limitation: Reported voice impersonation is not equivalent to forensic confirmation of AI use.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Other forms of AI, that are more domain-specific and specialized might help; those aren’t actually the ones getting the massive funding – or doing the massive harm.”
Our reading: Specialized, domain-specific AI might help with climate change, but it is not the kind getting massive funding or causing massive harm.
No horizon Evidence as of
Limitation: The annual target period is an explicit interpretation of an ongoing observation.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Altman had, much to my surprise, just echoed my longstanding position that current techniques alone would not be enough to get to AGI”
Our reading: Current AI techniques alone will not be enough to reach AGI.
No deadline stated Evidence as of
Limitation: There is no generally accepted empirical test for AGI [e1].
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“No such governance board ever materialized; if it does, you can bet it won’t have any teeth, unless there is a government mandate that it does.”
Our reading: The governance board Altman spoke of to The New Yorker in 2016 never materialized, and Marcus bets that if one does, it won't have any teeth unless a government mandate requires it to.
No deadline stated Evidence as of
Only if: if such a board materializes
Source: Scoop: What former employees of OpenAI are worried about
Assessment: UnresolvedSaid
Gary Marcus Observation
“I stressed the same in my Senate testimony, but nothing thus far in existing US law addresses this need.”
Our reading: So far, nothing in existing US law addresses the need for external governance under which companies cannot make decisions of potentially enormous magnitude on their own and external advisory bodies have a strong say in weighing risks against benefits.
No horizon Evidence as of
Limitation: A federal-only assessment would narrow the speaker's wording.
Source: Scoop: What former employees of OpenAI are worried about
Assessment: UnresolvedSaid
Gary Marcus Observation
“Companies like OpenAI that have talked big about investing in safety research and then stinted on their promises”
Our reading: OpenAI talked big about investing in safety research and then failed to deliver on those promises.
No horizon Evidence as of
Limitation: The packet does not establish which particular promise Marcus meant.
Assessment: UnresolvedSaid
Gary Marcus Observation
“And no, GPT-5 did not drop this week as many had hoped.”
Our reading: GPT-5 was not released in the week before 2024-08-18, despite many people hoping it would be.
No horizon Evidence as of
Limitation: The target-period evidence does not fully cover August 12–18, 2024 [e1, e2].
Source: What has and has not changed in the AI since the ChatGPT revolution? [Video]
Assessment: UnresolvedSaid
Gary Marcus Observation
“Sam Altman has acknowledged wanting to train on everyone's personal documents (Word files, email etc)”
Our reading: Sam Altman has acknowledged that he wants to train AI on everyone's personal documents, such as Word files and email.
No horizon Evidence as of
Limitation: The search cannot establish that no other qualifying statement exists.
Source: OpenAI: On a Path to Becoming The World’s Most Frightening Surveillance Company?
Assessment: UnresolvedSaid
Gary Marcus Observation
“OpenAI just put in money in a $60M fundraise with a Webcam company and is planning hardware joint venture with them.”
Our reading: OpenAI recently invested in a $60M fundraise for a webcam company and is planning a hardware joint venture with it.
No horizon Evidence as of
Limitation: The original financial report could not be read with read_page.
Source: OpenAI: On a Path to Becoming The World’s Most Frightening Surveillance Company?
Assessment: UnresolvedSaid
Gary Marcus Forecast
“LLMs are not AGI, and (on their own) never will be; scaling alone was never going to be enough.”
Our reading: LLMs on their own are not and will never be AGI.
No deadline stated Evidence as of
Limitation: The review cannot establish the absence of every possible demonstration.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“LLMs are not AGI, and (on their own) never will be; scaling alone was never going to be enough.”
Our reading: Scaling alone was never going to be enough to reach AGI.
No deadline stated Evidence as of
Limitation: Benchmark correlations do not establish whether scaling alone is sufficient for AGI [e1].
Assessment: UnresolvedSaid
Gary Marcus Observation
“The bubble has begun to burst.”
Our reading: As of late July 2024, the GenAI bubble has begun to burst.
No horizon Evidence as of
Limitation: A narrower claim about a technology-stock correction would be easier to support.
Assessment: UnresolvedSaid
Gary Marcus Observation
“I have long argued that Generative AI might be a dud; I just didn’t expect it to fade so fast.”
Our reading: Generative AI is fading fast, as of July 2024, and may turn out to be a dud.
No horizon Evidence as of
Limitation: The broad wording requires an evaluator-supplied reading.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Users have lost faith, clients have lost faith, VC’s have lost faith.”
Our reading: As of July 2024, users, clients, and venture capitalists have lost faith in generative AI.
No horizon Evidence as of
Limitation: General-public opinions cannot be treated as a representative survey of GenAI users.
Assessment: UnresolvedSaid
Gary Marcus Observation
“PS Microsoft Bing completely flubbed, 4/4, “people going yuck towards Generative AI”.”
Our reading: Microsoft Bing completely flubbed the prompt "people going yuck towards Generative AI", 4 out of 4 results.
No horizon Evidence as of
Limitation: Only one search query was run, so an independent record could still exist that this search missed.
Source: Most people don’t think GenAI has improved their productivity
Assessment: UnresolvedSaid
Gary Marcus Observation
“People are starting to realize all of the above.”
Our reading: People are starting to realize generative AI has plateaued, lacks a killer app, still hallucinates and errs, and has no moat.
No horizon Evidence as of
Limitation: The population and size of the claimed change are unspecified.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“That said, I am certain that the impending collapse won’t lead to the absolute death of AI. There is too much at stake.”
Our reading: The impending generative AI collapse will not lead to the absolute death of AI.
No deadline stated Evidence as of
Only if: after the generative AI collapse
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Forecast
“There can be no path to AGI without neurosymbolic AI.”
Our reading: AGI cannot be achieved without neurosymbolic AI.
No deadline stated Evidence as of
Limitation: Public demonstrations may not disclose enough implementation detail to classify a system's methods.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Observation
“Nobody has a moat”
Our reading: No generative AI company has a competitive moat.
No horizon Evidence as of
Limitation: The universal claim requires unusually broad evidence to support.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Forecast
“The neural network approach known as generative AI — especially its poster chatbot ChatGPT – has captured the world’s imagination, but will soon flame out.”
Our reading: Generative AI, especially ChatGPT, will soon flame out.
No deadline stated Evidence as of
Limitation: OpenAI's disclosure is self-reported.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Observation
“So for the most part, we’ve been stuck in a world in which all the major tech companies are building exactly the same thing (bigger and bigger LLMs based on more data), achieving almost exactly the same results (a passel of GPT-4 level models, hardly different from each other, all struggling with hallucinations and boneheaded errors), and hardly any investment in anything else.”
Our reading: Marcus says that, for the most part, all the major tech companies are building the same thing (bigger LLMs on more data) and getting almost the same results (GPT-4-level models that barely differ and all struggle with hallucinations and boneheaded errors), with hardly any investment in anything else.
No horizon Evidence as of
Limitation: Company evaluations are not a substitute for consistent independent comparisons.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Observation
“There is no killer app”
Our reading: Generative AI has no killer app.
No horizon Evidence as of
Limitation: Killer app remains a qualitative threshold.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Forecast
“The biggest intrinsic failings of generative AI have to do with reliability, in a way that I believe can never be solved, given their inherent nature.”
Our reading: Generative AI's reliability failings can never be solved, given its inherent nature.
No deadline stated Evidence as of
Limitation: The cited empirical study covers two benchmarks and a finite set of agents.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“Customers can easily switch to other providers, almost anytime they want, and for month there had been more and more alternatives, including Anthropic and Google, at more and more attractive prices, forcing OpenAI itself to cut prices just a few days ago.”
Our reading: Customers can easily switch LLM providers, and growing cheaper alternatives from Anthropic and Google forced OpenAI to cut prices in July 2024.
No horizon Evidence as of
Limitation: Company launch claims are not independent price-performance measurements.
Assessment: UnresolvedSaid
Gary Marcus Observation
“LLMs have just became exactly the commodity I predicted they would become, at the lowest possible price.”
Our reading: LLMs have become a commodity available for free.
No horizon Evidence as of
Limitation: Capability comparisons in the cited primary sources come from Meta.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Obviously, their only hope is to raise more cash, and they will certainly try.”
Our reading: OpenAI's only hope of survival is raising more cash, and it will try to.
No deadline stated Evidence as of
Limitation: This verdict retains both parts of the supplied claim; it does not silently reduce it to the fundraising prediction.
Assessment: UnresolvedSaid
Gary Marcus Observation
“And leaving more and more code writing to generative AI, which grasps syntax but not meaning, as a new benchmark from Stephen Wolfram shows, is not the answer, either.”
Our reading: Generative AI grasps syntax but not meaning, as shown by a new Stephen Wolfram benchmark.
No horizon Evidence as of
Limitation: This is an evidence gap, not a finding that Marcus misreported the benchmark.
Source: Don’t look up: the massive Microsoft/Crowdstrike data outage is a huge wake-up call
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Neurosymbolic AI has long been an underdog; in the end I expect it to come from behind and be essential.”
Our reading: Neurosymbolic AI will eventually come from behind and become essential.
No deadline stated Evidence as of
Limitation: The intended destination and meaning of essential require interpretation.
Source: Marcus goes gaga over Gates clip
Assessment: UnresolvedSaid
Gary Marcus Observation
“Others are bailing now, too. Or at least trying to subtly alter their positions, committing less to the unrealistic.”
Our reading: Prominent AI figures are now retreating from, or quietly softening, their unrealistic claims about LLMs and AGI.
No horizon Evidence as of
Limitation: Restricting others solely to the two named examples would produce a narrower assessment than the wording clearly warrants.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“LLMs are not AGI, and (on their own) never will be; scaling alone was never going to be enough.”
Our reading: Scaling alone is not enough to reach AGI.
No horizon Evidence as of
Limitation: Finite scaling experiments cannot establish what every larger system could achieve.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Anyone who can afford that can easily fill out some paperwork and run a few tests, just like I can pay a few bucks to get a driver’s license. It’s the least they can do.”
Our reading: Companies that can afford $100M+ training runs can easily afford SB-1047's paperwork and testing requirements.
No horizon Evidence as of
Limitation: The packet contains no documented estimate of the June proposal's incremental paperwork and testing costs matched to developers' available resources and existing commitments.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Every company that would be subject to the stronger set of regulations is valued in the billions.”
Our reading: Every company that would be subject to SB-1047's stronger regulations is valued in the billions of dollars.
No horizon Evidence as of
Limitation: Private-company valuations are intermittent and may refer to different share classes or transaction terms.
Assessment: UnresolvedSaid
Gary Marcus Observation
“It can do some things better, some worse, very few reliably.”
Our reading: Compared with a smart high schooler, GPT-4 can do some things better and some things worse, and it can do very few things reliably.
No horizon Evidence as of
Limitation: Both sources are company disclosures from 2023 and do not establish the performance of an identified June 2024 GPT-4 version [e1][e2].
Source: AGI by 2027?
Assessment: UnresolvedSaid
Gary Marcus Observation
“The double Y-axis makes no sense, and presupposes its own conclusion.”
Our reading: The double Y-axis on Aschenbrenner's graph makes no sense and presupposes its own conclusion.
No horizon Evidence as of
Limitation: The phrase makes no sense remains an evaluative judgment.
Source: AGI by 2027?
Assessment: UnresolvedSaid
Gary Marcus Observation
“In 2019 I pointed out through a series of examples on Twitter with GPT-2 that LLMs were subject to essentially the same critique: greedy with respect to data, uninterpretable, unreliable as reasoners, subject to hallucinations, etc, that they lacked world models (“they don't develop robust representations of *how events unfold over time*), and that they would be inadequate on their own.”
Our reading: In 2019, using a series of GPT-2 examples on Twitter, Marcus pointed out that LLMs were greedy with respect to data, uninterpretable, unreliable as reasoners, and subject to hallucinations, that they lacked world models, and that they would be inadequate on their own.
No horizon Evidence as of
Limitation: The reply does not document the boundaries or full contents of the claimed 2019 GPT-2 Twitter series [e1].
Assessment: UnresolvedSaid
Gary Marcus Observation
“Nowadays we both (unlike many other people) stress the absolutely essential nature of common sense, physical reasoning and world models, and the failure of current architectures to handle those well.”
Our reading: Marcus and LeCun, unlike many other people, now both stress that common sense, physical reasoning and world models are absolutely essential, and that current architectures fail to handle them well.
No horizon Evidence as of
Limitation: Public agreement does not establish who originated an idea or whether one speaker changed his position.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Worse, we are no closer to a solution to the alignment problem now than we were then.”
Our reading: There has been no progress toward solving the AI alignment problem between 2022 and mid-2024.
No horizon Evidence as of
Limitation: A narrower interpretation covering any improvement in present-model alignment would favor contradiction.
Assessment: UnresolvedSaid
Gary Marcus Observation
“But OpenAI has been and continues to be a California-registered 501(3)(c) nonprofit, pledged not to profits, but to humanity – and still getting tax-exempt status on the basis of their promise to deploy AI safely:”
Our reading: OpenAI is a registered nonprofit receiving tax-exempt status based on its promise to deploy AI safely.
No horizon Evidence as of
Limitation: The packet lacks dated records confirming continued California registration and federal exemption through May 30, 2024; this does not establish that either had lapsed.
Source: Disturbing and misleading efforts to defame Helen Toner through misdirection
Assessment: UnresolvedSaid
Gary Marcus Observation
“The 3.5 API (geared towards programmers as a backend tool) was available beforehand, but that’s not of course the consumer-directed product with gimmicks like word-by-word typing to give a false impression of humanness that ultimately went viral.”
Our reading: ChatGPT, unlike the earlier GPT-3.5 API, was a consumer product using gimmicks like word-by-word typing to give a false impression of humanness.
No horizon Evidence as of
Limitation: Earlier model availability rests on a contemporaneous community report, which does not identify an official product named GPT-3.5 API [e2].
Source: Disturbing and misleading efforts to defame Helen Toner through misdirection
Assessment: UnresolvedSaid
Gary Marcus Observation
“Fundamentally, the essay attacks a strawman, viz. that the firing of Altman was somehow precipitated by an immediate safety concern), thereby diverting from the board’s actual, stated concern (candor).”
Our reading: The Taylor/Summers essay attacks a strawman about safety, diverting from the board's actual concern of candor in firing Altman.
No horizon Evidence as of
Limitation: The complete essay was unavailable; the retrieved reporting cannot establish what the full text omitted.
Source: Disturbing and misleading efforts to defame Helen Toner through misdirection
Assessment: UnresolvedSaid
Gary Marcus Observation
“And oops, Grok’s basic understanding of time is shaky, too:”
Our reading: Grok has a shaky basic understanding of time.
No horizon Evidence as of
Limitation: A missing public test is not evidence that Grok performed well.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Musk is just building bigger and bigger LLMs, and Meta is mostly (despite LeCun’s reservations) mostly doing the same.”
Our reading: Musk's xAI is just building bigger LLMs, and Meta is mostly doing the same.
No horizon Evidence as of
Limitation: Public announcements do not provide a complete account of internal research.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“By clustering piles of similar things together in a giant n-dimensional space, they pretty much automatically become synonym and pastische machines, regurgitating a lot of words with slight paraphrases while adding conceptually little, and understanding even less.”
Our reading: Because LLMs cluster similar things in a high-dimensional space, they become synonym and pastiche machines that regurgitate with slight paraphrase, adding little and understanding less.
No horizon Evidence as of
Limitation: Verbatim recall and memorization from similar sequences do not establish how often models produce close paraphrases in ordinary use [e1, e3].
Assessment: UnresolvedSaid
Gary Marcus Observation
“It’s a pattern.”
Our reading: Sam Altman's lack of candor, seen in his statements to the Senate about OpenAI equity, his lies about Helen Toner and his lies about Scarlett Johansson, is a pattern.
No horizon Evidence as of
Limitation: The equity evidence does not establish whether the Senate answer was a misleading omission or material deception.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Gary Marcus Observation
“I probably don’t have to tell you, but that’s complete bullshit. And stupid, obviously refuted bullshit at that.”
Our reading: OpenAI's claim that its voice's resemblance to Scarlett Johansson was coincidental is false.
No horizon Evidence as of
Limitation: Earlier casting evidence is context outside the denial's target period.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Gary Marcus Observation
“They said it wasn’t intentional, but of course it was.”
Our reading: OpenAI intentionally made its voice resemble Scarlett Johansson's.
No horizon Evidence as of
Limitation: The primary production account comes from the company whose conduct is disputed.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Gary Marcus Observation
“I can’t tell you what happened to Sam’s caps lock, but obviously the claim that the resemblance to ScarJo was a “coincidence” was a lie. Sam knew perfectly well what the character sounded like.”
Our reading: Sam Altman knew the voice resembled Johansson's Her character, so the coincidence claim was a lie.
No horizon Evidence as of
Limitation: The public record is largely circumstantial and includes interested-party accounts.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Gary Marcus Observation
“Lots of quirky errors are already being reported, same as ever, like this reasoning error from Jane Rosenzweig:”
Our reading: Lots of quirky errors in GPT-4o are already being reported, same as ever, such as a reasoning error reported by Jane Rosenzweig.
No horizon Evidence as of
Limitation: Social-media text extraction omitted the underlying example.
Source: Hot take on OpenAI’s new GPT-4o
Assessment: UnresolvedSaid
Gary Marcus Observation
“They don’t have GPT-5 after 14 months of trying.”
Our reading: OpenAI does not have GPT-5 after 14 months of trying since GPT-4.
No horizon Evidence as of
Limitation: Internal completion status may not be public.
Source: Hot take on OpenAI’s new GPT-4o
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“LLMs confined to X tweets will be even less grounded in reality.”
Our reading: LLMs whose synthesis is confined to X tweets will be even less grounded in reality than LLMs in general.
No horizon Evidence as of
Limitation: This is an unresolved causal assessment, not a finding that Marcus was wrong.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“That means, of course, that OpenAI’s business model is threatened; same for Anthropic’s, Mistral’s, and so on.”
Our reading: Meta's free near-GPT-4 model threatens the business models of OpenAI, Anthropic, Mistral and similar companies.
No horizon Evidence as of
Limitation: Commercial threat is broader than benchmark performance.
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Whatever tiny moat there was between the companies will further diminish.”
Our reading: The small moat between LLM companies will further diminish.
No deadline stated Evidence as of
Limitation: The ordinary business meaning of moat is broader than model quality.
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“A billion robots with a superficial understanding of the physical world would be a recipe for disaster.”
Our reading: Deploying a billion robots with only superficial understanding of the physical world would lead to disaster.
No deadline stated Evidence as of
Only if: if robots have only superficial understanding of the physical world
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Khosla is a venture capitalist, and I think venture capitalists will have a huge role in whether or not we get to the promised land.”
Our reading: Marcus thinks venture capitalists will have a huge role in whether or not we get to the promised land of world-improving technology.
No deadline stated Evidence as of
Limitation: The destination and magnitude of influence remain qualitative.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Microsoft’s Mustafa Suleyman promised that hallucinations would be cured “soon”, yet my X feed is still filled with examples like these from Princeton Professor Aleksandra Korolova:”
Our reading: Although Mustafa Suleyman promised that hallucinations would be cured "soon", Marcus's X feed was still filled with examples of AI hallucinations, such as those from Princeton Professor Aleksandra Korolova.
No horizon Evidence as of
Limitation: Partial support is not a mixed outcome because no contradictory outcome evidence was found.
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“We won’t get to a billion personal robots if they are as dodgy as driverless cars, frequently working, yet stymied often enough by outliers that we can’t fully count on them.”
Our reading: A billion personal robots will not be deployed if robots are as unreliable with outliers as driverless cars are.
No deadline stated Evidence as of
Only if: if robots are as unreliable as driverless cars
Assessment: UnresolvedSaid
Gary Marcus Observation
“I think it’s fair to say that the multimodal stuff still is very buggy, having troubles with basics like the English alphabet.”
Our reading: OpenAI's multimodal capabilities remain very buggy as of April 2024, struggling even with basics like the English alphabet.
No horizon Evidence as of
Limitation: The assessment is limited by source-access failures.
Source: TED, AI, and the test of time
Assessment: UnresolvedSaid
Gary Marcus Observation
“I warned that LLMs would be used by bad actors, and they increasingly are.”
Our reading: LLMs are increasingly being used by bad actors.
No horizon Evidence as of
Limitation: Malicious use can remain undetected.
Source: TED, AI, and the test of time
Assessment: UnresolvedSaid
Gary Marcus Observation
“I am just not seeing that as OpenAI’s mission, anymore, especially when they have continued to fail to compensate most of the creators whose work they are drawing on, largely without consent.”
Our reading: OpenAI no longer appears to pursue its mission of ensuring AGI benefits all of humanity, and it has continued not to compensate most creators whose work it uses, largely without their consent.
No horizon Evidence as of
Limitation: The factual component is assessable in principle, but the required population and coverage records are missing.
Source: TED, AI, and the test of time