“Update: This is consistent with the finding of the Remote Labor Index that AI could only do about 2.5% of jobs, reported recently by the Washington Post.”
Our reading, the sentence that was checked:
Per the Remote Labor Index, AI can do only about 2.5% of jobs, consistent with LLMs adding little quantifiable value.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Let’s be honest, Generative AI isn’t going all that well, paragraph 5.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-lets-be-honest-generative-ai-isnt-0e7b76-a2
We recorded 7 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-01-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- finding of the Remote Labor Index
- AI could only do about 2.5% of jobs
- reported recently by the Washington Post
- consistent with
Our interpretation
- Because the sentence expressly attributes a finding to RLI, jobs means the commissioned projects in that benchmark.
- The percentage refers to the best tested agent's end-to-end completion rate, not every agent's rate or the share of all occupations that AI can perform.
- The target is the original RLI evaluation reported in 2025, rather than benchmark updates after the post.
- Consistent with is read as compatibility with limited autonomous performance, not proof of small aggregate economic benefits.
- The Washington Post attribution is checked separately from the primary numerical finding.
Counts as supported if: The original RLI report records approximately 2.5% as the highest end-to-end project completion rate among evaluated agents.
Counts as contradicted if: The original report gives a materially different leading completion rate, or the 2.5% figure measures something other than successful completion of benchmark projects.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The original RLI report, submitted on October 30, 2025, reports a 2.5% automation rate for its highest-performing tested agent on real-world projects evaluated for end-to-end performance [e1]. This supports the numerical claim within that benchmark's scope [e1].
Limitations
- The benchmark result does not establish the share of all occupations AI can perform or aggregate economic value from LLM use [e1].
- Support is limited to the original evaluation and its tested agents [e1].
- The supplied evidence does not verify the Washington Post's wording or attribution.
Evidence (1 item)
Supports the claim: Center for AI Safety and Scale AI / arXiv,
automation rate of 2.5%.
Why it matters here: The original report states exactly the cited leading automation rate for its real-world project benchmark. It measures successful end-to-end project completion, not the percentage of all jobs that AI can perform or the value of human–AI assistance.
- Where
- Abstract
- Publisher
- Center for AI Safety and Scale AI / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Let’s be honest, Generative AI isn’t going all that well
Update: This is consistent with the finding of the Remote Labor Index that AI could only do about 2.5% of jobs, reported recently by the Washington Post.
- Where
- paragraph 5
- Published
- 2026-01-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-lets-be-honest-generative-ai-isnt-0e7b76-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-lets-be-honest-generative-ai-isnt-0e7b76-a2.