Unresolved claims by Gary Marcus, page 5 of 8
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Unresolved claims by Gary Marcus (772)
The record was checked, and the evidence available by the cutoff does not settle it. Forecasts whose deadline has not arrived are listed apart, as not yet due.
Page 5 of 8: claims 401 to 500 of 772, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: UnresolvedSaid
Observation
“Even if you look at OpenAI’s own research post on o1, which introduced the newest putative law, it is immediately apparent that scaling test time compute works for some problems and not others.”
Our reading: OpenAI's own o1 research shows that scaling test-time compute helps on some problems but not others.
No horizon Evidence as of
Limitation: The conclusion depends on distinguishing model comparisons from compute-scaling experiments.
Assessment: UnresolvedSaid
Forecast
“Like the first generalization, the new ones will eventually run out.”
Our reading: The new scaling laws (post-training and inference-time) will eventually stop holding.
No deadline stated Evidence as of
Limitation: A narrower reading limited to particular scaling recipes would be easier to support.
Assessment: UnresolvedSaid
Observation
“Hundreds of billions of dollars were invested on the assumption that I was wrong, largely to the exclusion of investment in other approaches.”
Our reading: Hundreds of billions of dollars went into LLM scaling on the assumption that scaling would keep working, and alternative approaches got little investment.
No horizon Evidence as of
Limitation: The capex figure is a projection made in November 2024 for the full year, not a realized total.
Source: Satya Nadella and the three stages of scientific truth
Assessment: UnresolvedSaid
Observation
“Crucially, there has never been any principled argument that they would solve hallucinations or reasoning, or edge cases in open-ended worlds —or that synthetic data would suffice indefinitely to keep feeding the machine.”
Our reading: There has never been a principled argument that scaling would solve hallucinations, reasoning, or open-world edge cases, or that synthetic data would suffice indefinitely.
No horizon Evidence as of
Limitation: The claim concerns the existence of arguments, not whether those arguments ultimately proved correct.
Assessment: UnresolvedSaid
Observation
“Meanwhile, precious little investment has been made in other approaches.”
Our reading: Very little investment has gone into AI approaches other than LLMs.
No horizon Evidence as of
Limitation: The observation period is an explicit annual-reporting interpretation.
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Forecast
“Given the cost of chips, profits will be elusive.”
Our reading: Because chips are costly, LLM companies will struggle to make profits.
No deadline stated Evidence as of
Limitation: The cited financial evidence concerns one developer.
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Observation
“Just as I argued here in April 2024, LLMs have reached a point of diminishing returns.”
Our reading: LLMs have reached a point of diminishing returns from scaling.
No horizon Evidence as of
Limitation: The evidence lacks a comparable series of frontier training runs measuring capability gains against proportional increases in compute and data [e1, e2].
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Forecast
“LLMs such as they are, will become a commodity; price wars will keep revenue low.”
Our reading: LLMs such as they are will become a commodity, and price wars will keep revenue low.
No deadline stated Evidence as of
Limitation: The market study is a preprint.
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Conditional forecast
“If enthusiasm for GenAI dwindles and market valuations plummet, AI won’t disappear, and LLMs won’t disappear; they will still have their place as tools for statistical approximation.”
Our reading: If enthusiasm for GenAI dwindles and market valuations plummet, AI and LLMs will not disappear, and LLMs will still have their place as tools for statistical approximation.
No deadline stated Evidence as of
Only if: if enthusiasm for GenAI dwindles and market valuations plummet
Source: CONFIRMED: LLMs have indeed reached a point of diminishing returns
Assessment: UnresolvedSaid
Observation
“He then went on to extrapolate that it would be 10,000 times better in 4 years, never considering potential bottlenecks around data, or compute, nor inherent problems with LLMs.”
Our reading: When Musk extrapolated that AI would be 10,000 times better in 4 years, he did not consider potential bottlenecks around data or compute, or inherent problems with LLMs.
No horizon Evidence as of
Limitation: The only item is one news report. It covers parts of Musk's remarks, not the full session.
Assessment: UnresolvedSaid
Causal claim
“(Side bonus: GenAI uses a massive amount of power, to the point of keeping fossil fuels in vogue, and that’s good for oil-rich countries, even if GenAI never becomes reliable.)”
Our reading: Generative AI uses so much power that it keeps fossil fuels in demand, which benefits oil-rich countries whether or not GenAI ever becomes reliable.
No horizon Evidence as of
Limitation: The evidence covers data-centre demand in general, not generative AI specifically.
Assessment: UnresolvedSaid
Forecast
“It’s just fantasy to imagine selling 200 times as many humanoid robots in the nearish term when nobody knows how to build a single safe, reliable, generally useful humanoid right now, at any price.””
Our reading: Humanoid robots will not sell 200 times Roomba's roughly 50 million units in the near term.
No deadline stated Evidence as of
Limitation: The threshold inherits the approximation in Marcus's Roomba baseline.
Assessment: UnresolvedSaid
Forecast
“The same will be true for humanoid robots, and we aren’t going to see six humanoid robots for every car anytime soon.”
Our reading: Many people will not be able to afford or want humanoid robots, and there will not be six humanoid robots per car anytime soon.
No deadline stated Evidence as of
Limitation: Willingness to pay is different from inability to afford a purchase.
Assessment: UnresolvedSaid
Observation
“Meanwhile GPT-5 still hasn’t arrived, and nor has the much ballyhooed Sora.”
Our reading: As of October 2024, neither GPT-5 nor Sora has been released.
No horizon Evidence as of
Limitation: The required event-date restriction prevents the later launch announcements from serving as decisive evidence.
Assessment: UnresolvedSaid
Observation
“Those products have not by a longshot lived up to everyone’s expectations, but they do actually ship, and customers can try those products out and decide for themselves. It’s hard to call that smoke and mirrors.”
Our reading: OpenAI's products have not by a longshot lived up to everyone's expectations, but they do actually ship and customers can try them, so it is hard to call OpenAI smoke and mirrors.
No horizon Evidence as of
Limitation: Scoring only product access would omit an explicit part of the listed proposition.
Assessment: UnresolvedSaid
Observation
“But then again, their products have always been served with large side order of bullshit, like when Sam said, apparently jokingly, that “AGI has been achieved internally”. None of the robot stuff they promised in the Rubik’s cube campaign ever shipped.”
Our reading: OpenAI's products have always come with lots of hype, and none of the robotics promised in its Rubik's cube campaign ever shipped.
No horizon Evidence as of
Limitation: The distinction between research ambitions and product promises materially affects the assessment.
Assessment: UnresolvedSaid
Observation
“But the anti-neurosymbolic tradition is limited; its most visible manifestation, LLMs, has brought fame and money, but no robust solution to solving any particular problem with great reliability.”
Our reading: The anti-neurosymbolic tradition is limited. Its most visible manifestation, LLMs, has brought fame and money but no robust solution to solving any particular problem with great reliability.
No horizon Evidence as of
Limitation: High accuracy is evidence of useful capability and should not be dismissed merely because errors remain.
Assessment: UnresolvedSaid
Observation
“Almost every putative virtue of AI (aside from making a small number of people rich) has been promissory.”
Our reading: Almost every claimed benefit of AI, other than enriching a few people, has so far been only a promise.
No horizon Evidence as of
Limitation: The search identified examples and a clinical review, not a representative survey of AI promises.
Assessment: UnresolvedSaid
Observation
“Even though some of the drawbacks, which inherently are less dependent on reliably and tight causal reasoning, such as misinformation use, nonconsensual deepfake porn, ans deepfake kidnapping scams, are already here to say).”
Our reading: Some of AI's drawbacks, which are inherently less dependent on reliable and tight causal reasoning, such as misinformation, nonconsensual deepfake porn, and deepfake kidnapping scams, are already here to stay.
No horizon Evidence as of
Limitation: Reported voice impersonation is not equivalent to forensic confirmation of AI use.
Assessment: UnresolvedSaid
Observation
“Other forms of AI, that are more domain-specific and specialized might help; those aren’t actually the ones getting the massive funding – or doing the massive harm.”
Our reading: Specialized, domain-specific AI might help with climate change, but it is not the kind getting massive funding or causing massive harm.
No horizon Evidence as of
Limitation: The annual target period is an explicit interpretation of an ongoing observation.
Assessment: UnresolvedSaid
Forecast
“Altman had, much to my surprise, just echoed my longstanding position that current techniques alone would not be enough to get to AGI”
Our reading: Current AI techniques alone will not be enough to reach AGI.
No deadline stated Evidence as of
Limitation: There is no generally accepted empirical test for AGI [e1].
Assessment: UnresolvedSaid
Conditional forecast
“No such governance board ever materialized; if it does, you can bet it won’t have any teeth, unless there is a government mandate that it does.”
Our reading: The governance board Altman spoke of to The New Yorker in 2016 never materialized, and Marcus bets that if one does, it won't have any teeth unless a government mandate requires it to.
No deadline stated Evidence as of
Only if: if such a board materializes
Source: Scoop: What former employees of OpenAI are worried about
Assessment: UnresolvedSaid
Observation
“I stressed the same in my Senate testimony, but nothing thus far in existing US law addresses this need.”
Our reading: So far, nothing in existing US law addresses the need for external governance under which companies cannot make decisions of potentially enormous magnitude on their own and external advisory bodies have a strong say in weighing risks against benefits.
No horizon Evidence as of
Limitation: A federal-only assessment would narrow the speaker's wording.
Source: Scoop: What former employees of OpenAI are worried about
Assessment: UnresolvedSaid
Observation
“Companies like OpenAI that have talked big about investing in safety research and then stinted on their promises”
Our reading: OpenAI talked big about investing in safety research and then failed to deliver on those promises.
No horizon Evidence as of
Limitation: The packet does not establish which particular promise Marcus meant.
Assessment: UnresolvedSaid
Observation
“And no, GPT-5 did not drop this week as many had hoped.”
Our reading: GPT-5 was not released in the week before 2024-08-18, despite many people hoping it would be.
No horizon Evidence as of
Limitation: The target-period evidence does not fully cover August 12–18, 2024 [e1, e2].
Source: What has and has not changed in the AI since the ChatGPT revolution? [Video]
Assessment: UnresolvedSaid
Observation
“Sam Altman has acknowledged wanting to train on everyone's personal documents (Word files, email etc)”
Our reading: Sam Altman has acknowledged that he wants to train AI on everyone's personal documents, such as Word files and email.
No horizon Evidence as of
Limitation: The search cannot establish that no other qualifying statement exists.
Source: OpenAI: On a Path to Becoming The World’s Most Frightening Surveillance Company?
Assessment: UnresolvedSaid
Observation
“OpenAI just put in money in a $60M fundraise with a Webcam company and is planning hardware joint venture with them.”
Our reading: OpenAI recently invested in a $60M fundraise for a webcam company and is planning a hardware joint venture with it.
No horizon Evidence as of
Limitation: The original financial report could not be read with read_page.
Source: OpenAI: On a Path to Becoming The World’s Most Frightening Surveillance Company?
Assessment: UnresolvedSaid
Forecast
“LLMs are not AGI, and (on their own) never will be; scaling alone was never going to be enough.”
Our reading: LLMs on their own are not and will never be AGI.
No deadline stated Evidence as of
Limitation: The review cannot establish the absence of every possible demonstration.
Assessment: UnresolvedSaid
Forecast
“LLMs are not AGI, and (on their own) never will be; scaling alone was never going to be enough.”
Our reading: Scaling alone was never going to be enough to reach AGI.
No deadline stated Evidence as of
Limitation: Benchmark correlations do not establish whether scaling alone is sufficient for AGI [e1].
Assessment: UnresolvedSaid
Observation
“The bubble has begun to burst.”
Our reading: As of late July 2024, the GenAI bubble has begun to burst.
No horizon Evidence as of
Limitation: A narrower claim about a technology-stock correction would be easier to support.
Assessment: UnresolvedSaid
Observation
“I have long argued that Generative AI might be a dud; I just didn’t expect it to fade so fast.”
Our reading: Generative AI is fading fast, as of July 2024, and may turn out to be a dud.
No horizon Evidence as of
Limitation: The broad wording requires an evaluator-supplied reading.
Assessment: UnresolvedSaid
Observation
“Users have lost faith, clients have lost faith, VC’s have lost faith.”
Our reading: As of July 2024, users, clients, and venture capitalists have lost faith in generative AI.
No horizon Evidence as of
Limitation: General-public opinions cannot be treated as a representative survey of GenAI users.
Assessment: UnresolvedSaid
Observation
“PS Microsoft Bing completely flubbed, 4/4, “people going yuck towards Generative AI”.”
Our reading: Microsoft Bing completely flubbed the prompt "people going yuck towards Generative AI", 4 out of 4 results.
No horizon Evidence as of
Limitation: Only one search query was run, so an independent record could still exist that this search missed.
Source: Most people don’t think GenAI has improved their productivity
Assessment: UnresolvedSaid
Observation
“People are starting to realize all of the above.”
Our reading: People are starting to realize generative AI has plateaued, lacks a killer app, still hallucinates and errs, and has no moat.
No horizon Evidence as of
Limitation: The population and size of the claimed change are unspecified.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Conditional forecast
“That said, I am certain that the impending collapse won’t lead to the absolute death of AI. There is too much at stake.”
Our reading: The impending generative AI collapse will not lead to the absolute death of AI.
No deadline stated Evidence as of
Only if: after the generative AI collapse
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Forecast
“There can be no path to AGI without neurosymbolic AI.”
Our reading: AGI cannot be achieved without neurosymbolic AI.
No deadline stated Evidence as of
Limitation: Public demonstrations may not disclose enough implementation detail to classify a system's methods.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Observation
“Nobody has a moat”
Our reading: No generative AI company has a competitive moat.
No horizon Evidence as of
Limitation: The universal claim requires unusually broad evidence to support.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Forecast
“The neural network approach known as generative AI — especially its poster chatbot ChatGPT – has captured the world’s imagination, but will soon flame out.”
Our reading: Generative AI, especially ChatGPT, will soon flame out.
No deadline stated Evidence as of
Limitation: OpenAI's disclosure is self-reported.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Observation
“So for the most part, we’ve been stuck in a world in which all the major tech companies are building exactly the same thing (bigger and bigger LLMs based on more data), achieving almost exactly the same results (a passel of GPT-4 level models, hardly different from each other, all struggling with hallucinations and boneheaded errors), and hardly any investment in anything else.”
Our reading: Marcus says that, for the most part, all the major tech companies are building the same thing (bigger LLMs on more data) and getting almost the same results (GPT-4-level models that barely differ and all struggle with hallucinations and boneheaded errors), with hardly any investment in anything else.
No horizon Evidence as of
Limitation: Company evaluations are not a substitute for consistent independent comparisons.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Observation
“There is no killer app”
Our reading: Generative AI has no killer app.
No horizon Evidence as of
Limitation: Killer app remains a qualitative threshold.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Forecast
“The biggest intrinsic failings of generative AI have to do with reliability, in a way that I believe can never be solved, given their inherent nature.”
Our reading: Generative AI's reliability failings can never be solved, given its inherent nature.
No deadline stated Evidence as of
Limitation: The cited empirical study covers two benchmarks and a finite set of agents.
Source: AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
Assessment: UnresolvedSaid
Causal claim
“Customers can easily switch to other providers, almost anytime they want, and for month there had been more and more alternatives, including Anthropic and Google, at more and more attractive prices, forcing OpenAI itself to cut prices just a few days ago.”
Our reading: Customers can easily switch LLM providers, and growing cheaper alternatives from Anthropic and Google forced OpenAI to cut prices in July 2024.
No horizon Evidence as of
Limitation: Company launch claims are not independent price-performance measurements.
Assessment: UnresolvedSaid
Observation
“LLMs have just became exactly the commodity I predicted they would become, at the lowest possible price.”
Our reading: LLMs have become a commodity available for free.
No horizon Evidence as of
Limitation: Capability comparisons in the cited primary sources come from Meta.
Assessment: UnresolvedSaid
Forecast
“Obviously, their only hope is to raise more cash, and they will certainly try.”
Our reading: OpenAI's only hope of survival is raising more cash, and it will try to.
No deadline stated Evidence as of
Limitation: This verdict retains both parts of the supplied claim; it does not silently reduce it to the fundraising prediction.
Assessment: UnresolvedSaid
Observation
“And leaving more and more code writing to generative AI, which grasps syntax but not meaning, as a new benchmark from Stephen Wolfram shows, is not the answer, either.”
Our reading: Generative AI grasps syntax but not meaning, as shown by a new Stephen Wolfram benchmark.
No horizon Evidence as of
Limitation: This is an evidence gap, not a finding that Marcus misreported the benchmark.
Source: Don’t look up: the massive Microsoft/Crowdstrike data outage is a huge wake-up call
Assessment: UnresolvedSaid
Forecast
“Neurosymbolic AI has long been an underdog; in the end I expect it to come from behind and be essential.”
Our reading: Neurosymbolic AI will eventually come from behind and become essential.
No deadline stated Evidence as of
Limitation: The intended destination and meaning of essential require interpretation.
Source: Marcus goes gaga over Gates clip
Assessment: UnresolvedSaid
Observation
“Others are bailing now, too. Or at least trying to subtly alter their positions, committing less to the unrealistic.”
Our reading: Prominent AI figures are now retreating from, or quietly softening, their unrealistic claims about LLMs and AGI.
No horizon Evidence as of
Limitation: Restricting others solely to the two named examples would produce a narrower assessment than the wording clearly warrants.
Assessment: UnresolvedSaid
Causal claim
“LLMs are not AGI, and (on their own) never will be; scaling alone was never going to be enough.”
Our reading: Scaling alone is not enough to reach AGI.
No horizon Evidence as of
Limitation: Finite scaling experiments cannot establish what every larger system could achieve.
Assessment: UnresolvedSaid
Observation
“Anyone who can afford that can easily fill out some paperwork and run a few tests, just like I can pay a few bucks to get a driver’s license. It’s the least they can do.”
Our reading: Companies that can afford $100M+ training runs can easily afford SB-1047's paperwork and testing requirements.
No horizon Evidence as of
Limitation: The packet contains no documented estimate of the June proposal's incremental paperwork and testing costs matched to developers' available resources and existing commitments.
Assessment: UnresolvedSaid
Observation
“Every company that would be subject to the stronger set of regulations is valued in the billions.”
Our reading: Every company that would be subject to SB-1047's stronger regulations is valued in the billions of dollars.
No horizon Evidence as of
Limitation: Private-company valuations are intermittent and may refer to different share classes or transaction terms.
Assessment: UnresolvedSaid
Observation
“It can do some things better, some worse, very few reliably.”
Our reading: Compared with a smart high schooler, GPT-4 can do some things better and some things worse, and it can do very few things reliably.
No horizon Evidence as of
Limitation: Both sources are company disclosures from 2023 and do not establish the performance of an identified June 2024 GPT-4 version [e1][e2].
Source: AGI by 2027?
Assessment: UnresolvedSaid
Observation
“The double Y-axis makes no sense, and presupposes its own conclusion.”
Our reading: The double Y-axis on Aschenbrenner's graph makes no sense and presupposes its own conclusion.
No horizon Evidence as of
Limitation: The phrase makes no sense remains an evaluative judgment.
Source: AGI by 2027?
Assessment: UnresolvedSaid
Observation
“In 2019 I pointed out through a series of examples on Twitter with GPT-2 that LLMs were subject to essentially the same critique: greedy with respect to data, uninterpretable, unreliable as reasoners, subject to hallucinations, etc, that they lacked world models (“they don't develop robust representations of *how events unfold over time*), and that they would be inadequate on their own.”
Our reading: In 2019, using a series of GPT-2 examples on Twitter, Marcus pointed out that LLMs were greedy with respect to data, uninterpretable, unreliable as reasoners, and subject to hallucinations, that they lacked world models, and that they would be inadequate on their own.
No horizon Evidence as of
Limitation: The reply does not document the boundaries or full contents of the claimed 2019 GPT-2 Twitter series [e1].
Assessment: UnresolvedSaid
Observation
“Nowadays we both (unlike many other people) stress the absolutely essential nature of common sense, physical reasoning and world models, and the failure of current architectures to handle those well.”
Our reading: Marcus and LeCun, unlike many other people, now both stress that common sense, physical reasoning and world models are absolutely essential, and that current architectures fail to handle them well.
No horizon Evidence as of
Limitation: Public agreement does not establish who originated an idea or whether one speaker changed his position.
Assessment: UnresolvedSaid
Observation
“Worse, we are no closer to a solution to the alignment problem now than we were then.”
Our reading: There has been no progress toward solving the AI alignment problem between 2022 and mid-2024.
No horizon Evidence as of
Limitation: A narrower interpretation covering any improvement in present-model alignment would favor contradiction.
Assessment: UnresolvedSaid
Observation
“But OpenAI has been and continues to be a California-registered 501(3)(c) nonprofit, pledged not to profits, but to humanity – and still getting tax-exempt status on the basis of their promise to deploy AI safely:”
Our reading: OpenAI is a registered nonprofit receiving tax-exempt status based on its promise to deploy AI safely.
No horizon Evidence as of
Limitation: The packet lacks dated records confirming continued California registration and federal exemption through May 30, 2024; this does not establish that either had lapsed.
Source: Disturbing and misleading efforts to defame Helen Toner through misdirection
Assessment: UnresolvedSaid
Observation
“The 3.5 API (geared towards programmers as a backend tool) was available beforehand, but that’s not of course the consumer-directed product with gimmicks like word-by-word typing to give a false impression of humanness that ultimately went viral.”
Our reading: ChatGPT, unlike the earlier GPT-3.5 API, was a consumer product using gimmicks like word-by-word typing to give a false impression of humanness.
No horizon Evidence as of
Limitation: Earlier model availability rests on a contemporaneous community report, which does not identify an official product named GPT-3.5 API [e2].
Source: Disturbing and misleading efforts to defame Helen Toner through misdirection
Assessment: UnresolvedSaid
Observation
“Fundamentally, the essay attacks a strawman, viz. that the firing of Altman was somehow precipitated by an immediate safety concern), thereby diverting from the board’s actual, stated concern (candor).”
Our reading: The Taylor/Summers essay attacks a strawman about safety, diverting from the board's actual concern of candor in firing Altman.
No horizon Evidence as of
Limitation: The complete essay was unavailable; the retrieved reporting cannot establish what the full text omitted.
Source: Disturbing and misleading efforts to defame Helen Toner through misdirection
Assessment: UnresolvedSaid
Observation
“And oops, Grok’s basic understanding of time is shaky, too:”
Our reading: Grok has a shaky basic understanding of time.
No horizon Evidence as of
Limitation: A missing public test is not evidence that Grok performed well.
Assessment: UnresolvedSaid
Observation
“Musk is just building bigger and bigger LLMs, and Meta is mostly (despite LeCun’s reservations) mostly doing the same.”
Our reading: Musk's xAI is just building bigger LLMs, and Meta is mostly doing the same.
No horizon Evidence as of
Limitation: Public announcements do not provide a complete account of internal research.
Assessment: UnresolvedSaid
Causal claim
“By clustering piles of similar things together in a giant n-dimensional space, they pretty much automatically become synonym and pastische machines, regurgitating a lot of words with slight paraphrases while adding conceptually little, and understanding even less.”
Our reading: Because LLMs cluster similar things in a high-dimensional space, they become synonym and pastiche machines that regurgitate with slight paraphrase, adding little and understanding less.
No horizon Evidence as of
Limitation: Verbatim recall and memorization from similar sequences do not establish how often models produce close paraphrases in ordinary use [e1, e3].
Assessment: UnresolvedSaid
Observation
“It’s a pattern.”
Our reading: Sam Altman's lack of candor, seen in his statements to the Senate about OpenAI equity, his lies about Helen Toner and his lies about Scarlett Johansson, is a pattern.
No horizon Evidence as of
Limitation: The equity evidence does not establish whether the Senate answer was a misleading omission or material deception.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Observation
“I probably don’t have to tell you, but that’s complete bullshit. And stupid, obviously refuted bullshit at that.”
Our reading: OpenAI's claim that its voice's resemblance to Scarlett Johansson was coincidental is false.
No horizon Evidence as of
Limitation: Earlier casting evidence is context outside the denial's target period.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Observation
“They said it wasn’t intentional, but of course it was.”
Our reading: OpenAI intentionally made its voice resemble Scarlett Johansson's.
No horizon Evidence as of
Limitation: The primary production account comes from the company whose conduct is disputed.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Observation
“I can’t tell you what happened to Sam’s caps lock, but obviously the claim that the resemblance to ScarJo was a “coincidence” was a lie. Sam knew perfectly well what the character sounded like.”
Our reading: Sam Altman knew the voice resembled Johansson's Her character, so the coincidence claim was a lie.
No horizon Evidence as of
Limitation: The public record is largely circumstantial and includes interested-party accounts.
Source: The OpenAI board was right
Assessment: UnresolvedSaid
Observation
“Lots of quirky errors are already being reported, same as ever, like this reasoning error from Jane Rosenzweig:”
Our reading: Lots of quirky errors in GPT-4o are already being reported, same as ever, such as a reasoning error reported by Jane Rosenzweig.
No horizon Evidence as of
Limitation: Social-media text extraction omitted the underlying example.
Source: Hot take on OpenAI’s new GPT-4o
Assessment: UnresolvedSaid
Observation
“They don’t have GPT-5 after 14 months of trying.”
Our reading: OpenAI does not have GPT-5 after 14 months of trying since GPT-4.
No horizon Evidence as of
Limitation: Internal completion status may not be public.
Source: Hot take on OpenAI’s new GPT-4o
Assessment: UnresolvedSaid
Causal claim
“LLMs confined to X tweets will be even less grounded in reality.”
Our reading: LLMs whose synthesis is confined to X tweets will be even less grounded in reality than LLMs in general.
No horizon Evidence as of
Limitation: This is an unresolved causal assessment, not a finding that Marcus was wrong.
Assessment: UnresolvedSaid
Causal claim
“That means, of course, that OpenAI’s business model is threatened; same for Anthropic’s, Mistral’s, and so on.”
Our reading: Meta's free near-GPT-4 model threatens the business models of OpenAI, Anthropic, Mistral and similar companies.
No horizon Evidence as of
Limitation: Commercial threat is broader than benchmark performance.
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: UnresolvedSaid
Forecast
“Whatever tiny moat there was between the companies will further diminish.”
Our reading: The small moat between LLM companies will further diminish.
No deadline stated Evidence as of
Limitation: The ordinary business meaning of moat is broader than model quality.
Source: Groupthink versus Tulips. What will historians think of 2024?
Assessment: UnresolvedSaid
Conditional forecast
“A billion robots with a superficial understanding of the physical world would be a recipe for disaster.”
Our reading: Deploying a billion robots with only superficial understanding of the physical world would lead to disaster.
No deadline stated Evidence as of
Only if: if robots have only superficial understanding of the physical world
Assessment: UnresolvedSaid
Forecast
“Khosla is a venture capitalist, and I think venture capitalists will have a huge role in whether or not we get to the promised land.”
Our reading: Marcus thinks venture capitalists will have a huge role in whether or not we get to the promised land of world-improving technology.
No deadline stated Evidence as of
Limitation: The destination and magnitude of influence remain qualitative.
Assessment: UnresolvedSaid
Observation
“Microsoft’s Mustafa Suleyman promised that hallucinations would be cured “soon”, yet my X feed is still filled with examples like these from Princeton Professor Aleksandra Korolova:”
Our reading: Although Mustafa Suleyman promised that hallucinations would be cured "soon", Marcus's X feed was still filled with examples of AI hallucinations, such as those from Princeton Professor Aleksandra Korolova.
No horizon Evidence as of
Limitation: Partial support is not a mixed outcome because no contradictory outcome evidence was found.
Assessment: UnresolvedSaid
Conditional forecast
“We won’t get to a billion personal robots if they are as dodgy as driverless cars, frequently working, yet stymied often enough by outliers that we can’t fully count on them.”
Our reading: A billion personal robots will not be deployed if robots are as unreliable with outliers as driverless cars are.
No deadline stated Evidence as of
Only if: if robots are as unreliable as driverless cars
Assessment: UnresolvedSaid
Observation
“I think it’s fair to say that the multimodal stuff still is very buggy, having troubles with basics like the English alphabet.”
Our reading: OpenAI's multimodal capabilities remain very buggy as of April 2024, struggling even with basics like the English alphabet.
No horizon Evidence as of
Limitation: The assessment is limited by source-access failures.
Source: TED, AI, and the test of time
Assessment: UnresolvedSaid
Observation
“I warned that LLMs would be used by bad actors, and they increasingly are.”
Our reading: LLMs are increasingly being used by bad actors.
No horizon Evidence as of
Limitation: Malicious use can remain undetected.
Source: TED, AI, and the test of time
Assessment: UnresolvedSaid
Observation
“I am just not seeing that as OpenAI’s mission, anymore, especially when they have continued to fail to compensate most of the creators whose work they are drawing on, largely without consent.”
Our reading: OpenAI no longer appears to pursue its mission of ensuring AGI benefits all of humanity, and it has continued not to compensate most creators whose work it uses, largely without their consent.
No horizon Evidence as of
Limitation: The factual component is assessable in principle, but the required population and coverage records are missing.
Source: TED, AI, and the test of time
Assessment: UnresolvedSaid
Observation
“Another way to think about this (see third graph above) is that there about 5 - 7 recent models are on a par with GPT-4, but none is clearly and decisively ahead.”
Our reading: About 5 to 7 recent models are on par with GPT-4, but none is clearly ahead.
No horizon Evidence as of
Limitation: The earlier four-model list does not contradict a later approximate five-to-seven-model count.
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: UnresolvedSaid
Observation
“One more way to look at this, is this graph I just saw: enormous convergence on GPT-4 level performance, in multiple models released since, yet nothing decisively ahead.”
Our reading: Multiple models released since GPT-4 have converged on GPT-4-level performance with none decisively ahead.
No horizon Evidence as of
Limitation: A coding benchmark cannot establish general capability parity.
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: UnresolvedSaid
Conditional forecast
“If enthusiasm for GenAI dwindles and market valuations plummet, AI won’t disappear, and LLMs won’t disappear; they will still have their place as tools for statistical approximation.”
Our reading: If enthusiasm for GenAI dwindles and market valuations plummet, AI and LLMs will not disappear, and LLMs will still have their place as tools for statistical approximation.
No deadline stated Evidence as of
Only if: if enthusiasm for GenAI dwindles and market valuations plummet
Source: Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
Assessment: UnresolvedSaid
Forecast
“I couldn’t agree more. And parallel arguments could be made in many domains.”
Our reading: Marcus endorses the view that ChatGPT and similar tools will not themselves be virtual scientists anytime soon, and says parallel arguments could be made in many domains.
No deadline stated Evidence as of
Limitation: “Anytime soon” supplies no deadline; reaching the evaluation date cannot establish support.
Source: Superhuman AGI is not nigh
Assessment: UnresolvedSaid
Causal claim
“It also drives their greediness with data, and even the ethical choices their developers have been making.”
Our reading: Neural networks' trouble generalizing beyond their training space, at least in the configurations dominant over the last three decades, also drives their greediness with data and even the ethical choices their developers have been making.
No horizon Evidence as of
Limitation: The technical evidence concerns particular recurrent networks and tasks, and its link to data demand is tentative [e1].
Assessment: UnresolvedSaid
Forecast
“In short, diminishing returns are what we can expect.”
Our reading: Scaling data for current AI models will yield diminishing returns.
No deadline stated Evidence as of
Limitation: A narrower claim about a specified loss curve could be supported by individual experiments.
Assessment: UnresolvedSaid
Forecast
“Sooner or later, the exponential greed for data will exceed what is available.”
Our reading: The exponentially growing demand for training data will eventually exceed the data available.
No deadline stated Evidence as of
Limitation: Epoch's quantitative estimate covers public human-generated text, not all usable training data [e1].
Assessment: UnresolvedSaid
Observation
“an intellectual monoculture — with no alternative approach that is nearly as well-funded.”
Our reading: The AI field is an intellectual monoculture in which no alternative approach is nearly as well-funded as generative AI.
No horizon Evidence as of
Limitation: The annual reporting period is inferred.
Assessment: UnresolvedSaid
Observation
“At the same time, more optimistic predictions like these [from Reid Hoffman September 7, 2023] have not thus far borne out.”
Our reading: Reid Hoffman's optimistic September 2023 AI predictions had not come true as of March 2024.
No horizon Evidence as of
Limitation: The interview identifies the hallucination prediction but does not establish the complete set of predictions Marcus referenced. [e4]
Assessment: UnresolvedSaid
Observation
“So far most uses of LLMs have been for testing, not production-level systems.”
Our reading: Most uses of LLMs so far have been for testing rather than production systems.
No horizon Evidence as of
Limitation: Survey respondents are not interchangeable with projects or usage volume.
Assessment: UnresolvedSaid
Causal claim
“They are inherent to how the system makes predictions, and to the fact that LLMs themselves don’t have the wherewithal to look things up and reason over them.”
Our reading: Hallucinations are inherent to how LLMs make predictions and to their inability to look things up and reason over them.
No horizon Evidence as of
Limitation: A conditional mathematical result is not a universal result about deployed systems.
Assessment: UnresolvedSaid
Causal claim
“Meanwhile hype itself is becoming a kind of risk factor. The more hype, the bigger the fall, if expectations aren’t met.”
Our reading: Hype around generative AI is a risk factor: the more hype, the bigger the fall if expectations are not met.
No horizon Evidence as of
Limitation: The closest paper covers AI broadly and a particular national market.
Assessment: UnresolvedSaid
Observation
“Thus far, the closest thing to a killer app is probably coding assistace, but the income there isn’t massive enough to cover costs of the chips, legal etc.”
Our reading: Coding assistance is probably the closest thing to a generative AI killer app, but its income is not large enough to cover chip, legal and other costs.
No horizon Evidence as of
Limitation: The available financial reporting is secondary and concerns one product.
Assessment: UnresolvedSaid
Forecast
“Core problems like hallucinations aren’t going way.”
Our reading: Core generative AI problems like hallucinations are not going away.
No deadline stated Evidence as of
Limitation: The cited persistence observation is from September 2025, not a complete audit of every system available at the evaluation date.
Assessment: UnresolvedSaid
Causal claim
“(Not easy. With everyone building similar models, there is no moat, hence lots of competition on price).”
Our reading: Because everyone is building similar models, generative AI companies have no moat, leading to heavy price competition.
No horizon Evidence as of
Limitation: Observed convergence does not establish universal substitutability.
Assessment: UnresolvedSaid
Observation
“$50B in, $3B out. That’s obviously not sustainable.”
Our reading: Generative AI investment of about $50B against $3B in revenue is obviously not sustainable.
No horizon Evidence as of
Limitation: The repeated estimate is not an independent validation of the amounts, their coverage or whether revenue was realized or annualized. [e1]
Assessment: UnresolvedSaid
Observation
“Most of those who invested in driverless cars lost money, unless they invested early and cashed out, before patience began to run out.”
Our reading: Most of those who invested in driverless cars lost money, unless they invested early and cashed out before patience began to run out.
No horizon Evidence as of
Limitation: The evidence covers selected investments, not a representative population.
Source: The Second Worst $100B Investment in The History of AI?
Assessment: UnresolvedSaid
Observation
“There may be a payoff someday, but thus far there has not been a lot.”
Our reading: As of March 2024, investment in driverless cars has not produced much payoff.
No horizon Evidence as of
Limitation: The company examples are not a representative industry return calculation.
Source: The Second Worst $100B Investment in The History of AI?
Assessment: UnresolvedSaid
Observation
“(OpenAI even brazenly asked the British goverment to exempt them from copyright laws, wanting all the gain at none of the cost.)”
Our reading: OpenAI asked the British government to exempt it from copyright laws.
No horizon Evidence as of
Limitation: The official submission required an accessible mirror.
Source: The race between positive and negative applications of Generative AI is on – and not looking pretty
Assessment: UnresolvedSaid
Observation
“Translation: But I do have equity in YCombinator and YCombinator has equity in OpenAI. It’s probably worth a lot, but you wouldn’t like me as much if I acknowledged that, so I won’t. Also, did I mention I own the OpenAI venture fund? Nope, guess not, sorry.”
Our reading: In his May 2023 Senate testimony, Sam Altman left out his indirect OpenAI stake through Y Combinator and his ownership of the OpenAI venture fund.
No horizon Evidence as of
Limitation: This is an evidentiary hold, not a finding that either interest was absent.
Source: OpenAI’s Lies and Half-Truths
Assessment: UnresolvedSaid
Observation
“Two clearest examples so far this year, are the now retracted paper with the above-mentioned rat and a new lithium battery paper, with an opening sentence clearly written by an LLM.”
Our reading: In 2024 so far, a now-retracted paper with an AI-generated rat image and a lithium battery paper with an LLM-written opening sentence are the clearest examples of GenAI contaminating science.
No horizon Evidence as of
Limitation: The factual examples are strongly corroborated, but only the rat example was verified within the target period.
Source: The Year of the Big-Balled Rat
Assessment: UnresolvedSaid
Causal claim
“Musk's lawsuit, combined with OpenAI's bizarre arrangement with Microsoft in which the continuation of an exclusive license rests on whether or not OpenAI has achieved something that is difficult to define (AGI) has left OpenAI in the position of needing skeptical experts to deflate their own hype.”
Our reading: Musk's lawsuit plus the AGI-contingent Microsoft license have put OpenAI in the position of needing skeptical experts to deflate its own hype.
No horizon Evidence as of
Limitation: A practical incentive is easier to establish than necessity.
Source: The Irony and the Agony of Elon Musk's Lawsuit against OpenAI
Assessment: UnresolvedSaid
Observation
“I take the recital of facts in Musk’s lawsuit to be largely accurate. It is consistent with my own observations and with earlier reporting by The New York Times and The New Yorker.”
Our reading: The facts set out in Musk's lawsuit against OpenAI are largely accurate.
No horizon Evidence as of
Limitation: The original state complaint and later federal pleadings are different documents.
Source: Elon Musk sues OpenAI