“As it happens, large language models are also wicked hard to update, typically requiring full retraining, sometimes taking weeks or months.”
Our reading, the sentence that was checked:
LLMs are very hard to update, usually needing full retraining that can take weeks or months.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Is ChatGPT Really a “Code Red” for Google Search?, paragraph 52.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-is-chatgpt-really-a-code-red-for-c9a962-a1
We recorded 13 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-12-30. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- large language models
- wicked hard to update
- typically requiring full retraining
- sometimes taking weeks or months
Our interpretation
- Update means refreshing stored factual knowledge, as indicated by the surrounding comparison with databases and outdated information.
- Full retraining ordinarily means a new comprehensive training run, rather than a targeted factual edit.
- Typically concerns usual practice or requirements; one successful editing demonstration cannot establish what was usual.
- The ongoing observation is assessed for December 1–30, 2022, the current month before publication. Earlier experiments provide context rather than direct measurements of December practice.
Counts as supported if: Representative records show that factual updates to large language models usually required comprehensive retraining in December 2022, with documented update runs lasting weeks or months.
Counts as contradicted if: Representative records show that effective factual updates usually used incremental training or editing without comprehensive retraining, or otherwise directly refute the claimed typical requirement.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Months-long initial training is documented, and factual editing methods already existed. Neither establishes what updating a large model typically required in December 2022. Editing demonstrations weaken a claim of universal necessity, but Marcus said typically. Treating initial training duration as update duration would overstate the supporting evidence.
Limitations
- Most relevant records describe research methods rather than representative deployment practice.
- The scope of an update and the meaning of full retraining are not specified.
Evidence (3 items)
Context only: arXiv,
model-editing task
Why it matters here: The paper demonstrates targeted factual updates without a comprehensive retraining run. It challenges any absolute necessity claim, but does not establish typical December production practice.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv / BigScience Workshop,
about 3.5 months
Why it matters here: A large model's initial training did take months. This does not show that updating its factual knowledge required repeating that run.
- Publisher
- arXiv / BigScience Workshop
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Association for Computational Linguistics,
Continual PostTraining
Why it matters here: The research develops incremental extension of a model across domain corpora. Its few-shot domain-learning results do not measure routine updates to current factual knowledge across large deployed models.
- Publisher
- Association for Computational Linguistics
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
What would settle it
A representative historical record of December 2022 factual-knowledge updates, identifying update scope, training method, effectiveness, and elapsed training time.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Is ChatGPT Really a “Code Red” for Google Search?
As it happens, large language models are also wicked hard to update, typically requiring full retraining, sometimes taking weeks or months.
- Where
- paragraph 52
- Published
- 2022-12-30
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-is-chatgpt-really-a-code-red-for-c9a962-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-is-chatgpt-really-a-code-red-for-c9a962-a1.