“The results are disappointing because the underlying tech is unreliable, And that’s been obvious from the start.”
Our reading, the sentence that was checked:
Generative AI results are disappointing because the underlying technology is unreliable, which has been obvious from the start.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Three years on, ChatGPT still isn't what it was cracked up to be – and it probably never will be, paragraph 22.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-three-years-on-chatgpt-still-isnt-e70f1d-a1
We recorded 30 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-11-29. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- The results are disappointing
- because the underlying tech is unreliable
- obvious from the start
Our interpretation
- The surrounding discussion makes business adoption and practical returns the relevant results.
- The ordinary reading is that unreliable output materially contributes to disappointing results. The statement does not expressly claim that unreliability is the sole cause.
- I interpret 'obvious from the start' as publicly documented when ChatGPT launched, rather than universally recognized.
- The period runs from ChatGPT's launch through publication because the statement explicitly connects present results to the beginning.
Counts as supported if: Deployment research links unreliable output to lost productivity or substantial rework, business surveys identify reliability as an adoption barrier, and contemporaneous launch documentation acknowledges unreliable answers.
Counts as contradicted if: Relevant deployment research finds reliability has no material adverse effect on results, or contemporaneous records establish that the alleged launch limitations were absent.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence supports unreliability as a material contributor to disappointing practical results. A deployment experiment links wasted time to unreliable generations, and a professional-services survey identifies accuracy, rework and unreliable outputs as obstacles. OpenAI documented incorrect answers at launch. This supports the contributing-cause reading; it does not establish that unreliability dominates other explanations for weak aggregate demand.
Limitations
- The studies cover particular occupations and deployments.
- Business surveys report perceptions rather than independently measured causal effects.
- The result depends on reading 'because' as a material contributing cause.
Evidence (3 items)
Supports the claim: OpenAI,
incorrect or nonsensical answers.
Why it matters here: OpenAI acknowledged unreliable answers at launch. This supports the claim that the problem was publicly knowable from the beginning.
- Where
- Limitations
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: METR; arXiv,
developers accept <<44%
Why it matters here: The February–June experiment found a 19% slowdown with AI assistance. Its factor analysis identifies unreliable generations and time spent reviewing or rejecting them as contributors, although the reliability mechanism itself was not separately randomized.
- Publisher
- METR; arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: Boston Consulting Group,
substantial rework.
Why it matters here: The late-2024 survey of 300 US professionals connects modest benefits with rework and identifies unreliable outputs as a challenge for 30% of general-tool users. It also identifies privacy, customization and training barriers, limiting any claim that technical reliability alone explains adoption.
- Publisher
- Boston Consulting Group
- Kind of source
- Independent measurement
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Three years on, ChatGPT still isn't what it was cracked up to be – and it probably never will be
The results are disappointing because the underlying tech is unreliable, And that’s been obvious from the start.
- Where
- paragraph 22
- Published
- 2025-11-29
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-three-years-on-chatgpt-still-isnt-e70f1d-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-three-years-on-chatgpt-still-isnt-e70f1d-a1.