“LLMs are sold as “AI,” a technology best-known for automating things, yet they can’t be trusted to run anything on their own.”
Our reading, the sentence that was checked:
LLMs cannot be trusted to run anything on their own.
Assessment: Unresolved Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, AI Doesn't Have ROI, paragraph 151.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-ai-doesnt-have-roi-f1ac56-a2
We recorded 24 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-06-02. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- LLMs
- can’t be trusted
- to run anything on their own
Our interpretation
- The plain reading is a universal claim about practical autonomous operation, not a requirement that software never make an error.
- A counterexample must reliably execute a defined workflow without case-by-case human approval or correction. Ordinary setup, maintenance and monitoring are allowed.
- Marketing a workflow as automated or recording unanswered conversations as resolved does not itself establish reliability.
- For this ongoing capability observation, use May 2026, the latest complete month before the post. No reporting cadence is specified, so this period choice is an evaluator interpretation.
Counts as supported if: Broad operational evidence establishes that no relevant class of defined LLM workflow meets a reasonable task-specific reliability standard without case-by-case human supervision.
Counts as contradicted if: At least one production workflow in the target period demonstrably operates without case-by-case human supervision and meets a documented, task-appropriate standard for successful completion and errors.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Intercom's May 8, 2026 account says Fin freed capacity while staff continued substantial support work; it provides no verified autonomous success, error or intervention rates [e8]. A customer's May 15 post says Fin counts some conversations as assumed resolutions when customers stop replying, without establishing that their problems were solved [e9]. Neither item establishes a reliable autonomous workflow in May 2026 or shows that no such workflow could meet the contract's standard [e8, e9].
Limitations
- The evidence consists of a company account and customer testimony, not an independent operational audit.
- Neither item supplies total attempts, independently checked outcomes, errors, escalations and case-by-case human interventions for May 2026.
- The claim specifies no task scope or reliability threshold; the contract selects May 2026 as the target period.
Evidence (2 items)
Context only: Intercom,
Fin had freed up significant capacity
Why it matters here: Intercom describes an operating support arrangement with useful AI delegation and substantial continuing human work. The account supplies neither a verified autonomous success rate nor enough detail to establish that every delegated task requires supervision.
- Publisher
- Intercom
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Intercom Community; Christian Osmundsen, Deliverect,
an assumed resolution
Why it matters here: The customer describes why reported resolution rates may overstate verified success. This is a measurement caution, not proof that Fin fails or that autonomous LLM operation is impossible.
- Publisher
- Intercom Community; Christian Osmundsen, Deliverect
- Kind of source
- Secondary summary
- Describes events of
- Public from
What would settle it
Validated May 2026 records for a defined production workflow, reporting total attempts, successful outcomes, errors, escalations and human interventions, could establish a counterexample if it met a documented, task-appropriate reliability standard without case-by-case supervision. Supporting the universal claim would require broad operational evidence establishing that no relevant class of workflow met that standard.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, AI Doesn't Have ROI
LLMs are sold as “AI,” a technology best-known for automating things, yet they can’t be trusted to run anything on their own.
- Where
- paragraph 151
- Published
- 2026-06-02
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-ai-doesnt-have-roi-f1ac56-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.