“So they forever play a game of catchup. Using massive numbers of humans (exact numbers unknown), GenAI companies perpetually put band-aids that they never disclose on top of their LLMs, based on human corrections, fixing each new problem as it arises. But the cycle repeats, endlessly.”
Our reading:
GenAI companies will endlessly play catch-up, using many humans to apply undisclosed patches to their LLMs as each new problem arises.
Assessment: Unresolved
What happened
Did the forecast come true?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
OpenAI’s October 7, 2026 disclosure reports safety-training updates informed by real-world use, but does not quantify the human labor involved [e5]. Anthropic’s August 28, 2026 study reports automated fixes for 10 alignment-failure categories on selected benchmarks; it does not demonstrate sustained handling of novel failures across representative production deployments [e6]. Anthropic’s September 17, 2025 postmortem disclosed fixes for three infrastructure bugs, before the January 2026 forecast [e4]. These records neither establish endless dependence on substantial human correction nor demonstrate the sustained industry transition required for early contradiction [e4, e5, e6].
Caveats
- The evidence consists of company disclosures, without independent longitudinal deployment audits.
- Human labor devoted specifically to correcting new failures is not quantified.
- The September 2025 postmortem predates the forecast and does not establish how routinely or comprehensively interventions are disclosed.
- The automated-correction study does not establish durable production performance.
- Finite observations cannot establish endless recurrence.
Evidence (3 items)
Context only: Anthropic,
We've now resolved these issues
Why it matters: The postmortem describes infrastructure failures investigated after user reports and publicly explains corrective work. It qualifies the opacity allegation, but predates the forecast and provides no inventory of undisclosed interventions.
- Where
- Opening paragraph
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
Context only: OpenAI,
updated the safety training
Why it matters: This documents continued adaptation during the forecast period. It does not establish substantial corrective headcount, the completeness of disclosure, or perpetual dependence on human fixes.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
Context only: Anthropic,
Claude found fixes
Why it matters: Automated generation of effective fixes is material counterevidence to permanent dependence on human correction. The authors limit their findings to selected evaluations and do not establish durable production performance or an industry-wide escape from the cycle.
- Where
- Results paragraph before 'Can Claude post-train a production-grade model for better alignment?'
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
How we judged it
- Kind of claim
- Forecast
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The plain reading predicts a continuing industry dependence on substantial human work to identify and correct new failures, with the routine corrective process largely hidden from users.
- Band-aids can include changes to training, model weights, prompts, retrieval, safeguards, or other system components.
- The disclosure allegation concerns the general visibility of routine corrective work. One published fix does not by itself refute it; isolated disclosures and promises of transparency must be distinguished from comprehensive practice.
- Forever and endlessly supply no deadline. Repeated failures during a finite period cannot establish permanent dependence.
- Massive numbers means substantial human labor devoted to this corrective process. The speaker supplies neither a headcount threshold nor a denominator.
Right if: Longitudinal audits show continued substantial human correction of newly encountered failures and limited disclosure of routine interventions across representative companies. Such evidence supports the observed pattern but cannot conclusively establish that it continues forever.
Wrong if: Independently verified, sustained changes across representative deployed systems demonstrate that the industry has escaped dependence on recurring human correction of new failures. Comprehensive disclosure records could separately contradict the opacity component; isolated announcements cannot settle the compound forecast.
We wrote this test after the outcome was known.
What would settle it
Independent longitudinal audits showing a sustained transition across representative production deployments to handling novel failures without recurring substantial human correction would meet the early-contradiction rule. Comprehensive disclosure records could separately settle the opacity component. Continued corrective work could support an observed pattern, but cannot prove endless recurrence.
We found 9 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Why ChatGPT can’t be trusted with breaking news, paragraph 13.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2026-01-03.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Why ChatGPT can’t be trusted with breaking news
So they forever play a game of catchup. Using massive numbers of humans (exact numbers unknown), GenAI companies perpetually put band-aids that they never disclose on top of their LLMs, based on human corrections, fixing each new problem as it arises. But the cycle repeats, endlessly.
- Where
- paragraph 13
- Published
- 2026-01-03
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.