“At the task level, analyses of workplace automation show that even state-of-the-art systems can reliably perform only a small fraction of the tasks required in typical human occupations (Eloundou et al., 2023; Felten et al., 2023), despite high scores on idealized benchmarks.”
Our reading, the sentence that was checked:
Even state-of-the-art AI systems can reliably perform only a small fraction of tasks in typical human occupations.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated, paragraph 21.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-rumors-of-agis-arrival-have-been-981d53-a1
We recorded 19 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-02-17. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- even state-of-the-art systems
- reliably perform
- only a small fraction
- tasks required in typical human occupations
- Eloundou et al., 2023; Felten et al., 2023
Our interpretation
- This describes capabilities at publication, rather than only the capabilities of models studied in 2023.
- Because occupational evaluations have no common reporting quarter, the target period covers the latest evaluation cycle: 2025 through February 17, 2026.
- Reliably perform means complete tasks to an acceptable professional standard with consistent success, rather than merely assist workers or potentially shorten task duration.
- Small fraction means a clearly small minority. No numerical cutoff is supplied or imposed.
- Typical occupations requires coverage representative of occupational tasks, rather than only selected digital projects.
Counts as supported if: Representative evaluations of contemporary frontier systems show reliable completion of only a small share of occupational tasks.
Counts as contradicted if: Representative evaluations show reliable completion of a broad share of occupational tasks inconsistent with a small minority.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The cited papers measure exposure, not reliable task completion. Later evidence cuts in different directions: RLI finds very low end-to-end automation, while GPT-5.2 performs strongly on selected GDPval tasks. Neither supplies the representative occupational denominator and consistent reliability measure needed to establish or refute the broad claim. A citation mismatch alone does not prove the underlying claim false.
Limitations
- Digital projects and selected knowledge-work tasks do not represent every occupation.
- Different task sizes and evaluation methods prevent direct comparison of the reported percentages.
Evidence (4 items)
Context only: arXiv; Tyna Eloundou and colleagues,
potential economic impact
Why it matters here: The cited paper estimates exposure, including potential assistance. It does not measure reliable completion by systems current in February 2026.
- Publisher
- arXiv; Tyna Eloundou and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Ed Felten, Manav Raj and Robert Seamans,
“exposure” to AI.
Why it matters here: The methodology deliberately leaves open whether exposure produces augmentation or substitution. It cannot establish a fraction of tasks reliably performed by contemporary systems.
- Publisher
- arXiv; Ed Felten, Manav Raj and Robert Seamans
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Remote Labor Index research team,
automation rate of 2.5%.
Why it matters here: This supports limited end-to-end automation on the sampled remote-work projects. Those projects do not provide a representative denominator for all tasks in typical occupations.
- Where
- Abstract
- Publisher
- arXiv; Remote Labor Index research team
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: OpenAI,
70.9%
Why it matters here: This challenges a blanket characterization of poor professional-task performance. A win-or-tie rate on selected, well-specified knowledge-work tasks is not a representative occupational completion rate or a repeated-reliability measure.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
A representative, occupationally weighted evaluation of systems available at publication, measuring repeated completion to professional standards under a specified level of human assistance.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated
At the task level, analyses of workplace automation show that even state-of-the-art systems can reliably perform only a small fraction of the tasks required in typical human occupations (Eloundou et al., 2023; Felten et al., 2023), despite high scores on idealized benchmarks.
- Where
- paragraph 21
- Published
- 2026-02-17
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-rumors-of-agis-arrival-have-been-981d53-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-rumors-of-agis-arrival-have-been-981d53-a1.