“In 2029, AI will not be able to reliably construct bug-free code of more than 10,000 lines from natural language specification or by interactions with a non-expert user.”
Our reading, the sentence that was checked:
In 2029, AI will not be able to reliably build bug-free code over 10,000 lines from a natural-language spec or from talking with a non-expert user, not counting glued-together library code.
Assessment: Unresolved Not yet due on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Dear Elon Musk, here are five things you might want to consider about AGI, prediction 4 of 5.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-dear-elon-musk-here-are-five-things-082354-a1
We recorded 21 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2022-05-31. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “In 2029”
- Deadline state when assessed
- Not yet due
- Deadline state on 2026-10-29
- Not yet due (the release’s reference date)
Conditions
- from natural language spec or non-expert interaction (qualifier, stated by the speaker)
- gluing existing library code doesn't count (qualifier, stated by the speaker)
From the speaker
- In 2029
- AI will not be able to reliably construct bug-free code
- more than 10,000 lines
- from natural language specification or by interactions with a non-expert user
- Gluing together code from existing libraries doesn’t count.
Our interpretation
- The deadline is December 31, 2029, allowing the whole named year.
- Reliably means repeatable success across varied substantive programming tasks, rather than one selected demonstration. Marcus supplies no numerical success rate.
- Bug-free means correctly implementing the requirements without functional defects. Passing a finite test suite is evidence of correctness, but does not by itself establish this.
- Count substantive newly generated source code, excluding padding and pre-existing library code. Marcus does not specify a line-count convention.
- Either input route qualifies. An expert may supply the natural-language specification under the first route; expert implementation or repair would not establish the AI's capability unaided.
- The input restrictions define the task, rather than a conditional forecast requiring a separate antecedent.
- This assessment concerns the individual programming prediction, not winning the broader three-of-five AGI bet.
Counts as supported if: Evaluations during 2029, covering leading available systems and varied qualifying tasks, consistently find that systems cannot reliably produce correct new programs exceeding 10,000 substantive source lines under either permitted input route. Mere absence of a public demonstration is insufficient.
Counts as contradicted if: A system repeatedly produces correct new programs exceeding 10,000 substantive source lines from natural-language specifications or non-expert interaction, with independently assessed correctness and without counting library assembly as the implementation.
Prospective: these criteria were frozen before the deadline, while the outcome was not yet public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Not yet due
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The deadline is December 31, 2029. The compiler demonstration shows substantial progress beyond the size threshold, but its disclosed defects prevent it from establishing reliable bug-free construction. The verification work proves selected properties of existing software under stated assumptions. Neither meets the early-resolution rule. Present limitations also cannot establish what AI will be unable to do in 2029.
Limitations
- The cited demonstrations are developer reports, not independent evaluations satisfying the whole contract.
- The search cannot establish that no qualifying system exists.
- No numerical reliability threshold was supplied; none is invented here.
Evidence (2 items)
Context only: Anthropic,
New features and bugfixes frequently broke existing functionality.
Why it matters here: Anthropic reported a newly built 100,000-line compiler, exceeding the forecast's size threshold. The same report describes defects, regressions and substantial expert work on testing, so it does not establish reliable bug-free construction under the complete criteria.
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Anthropic,
“Formally verified” does not mean “bug-free.”
Why it matters here: The report describes proving security properties for existing software under explicit machine and input assumptions. This is relevant progress, but it does not demonstrate repeated generation of qualifying new programs with complete correctness against natural-language requirements.
- Where
- Limitations
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
A reproducible independent evaluation showing repeated qualifying generation could contradict the forecast early. Otherwise, broad evaluations during 2029 must assess the full task, with disclosed success rates, substantive line counts, intervention logs and correctness evidence.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Dear Elon Musk, here are five things you might want to consider about AGI
In 2029, AI will not be able to reliably construct bug-free code of more than 10,000 lines from natural language specification or by interactions with a non-expert user.
- Where
- prediction 4 of 5
- Published
- 2022-05-31
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-dear-elon-musk-here-are-five-things-082354-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-dear-elon-musk-here-are-five-things-082354-a1.