“One that you can just ask to do some shit and know it’ll do it, and it’d also be very obvious what it could actually do, which is not the case right now. A better ChatGPT would quite literally be a different product.”
Our reading, the sentence that was checked:
ChatGPT currently cannot be relied on to do what it's asked and its capabilities are unclear; a better ChatGPT would be a different product.
Assessment: Unresolved Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Never Forget What They've Done, paragraph 25.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-never-forget-what-theyve-done-1a44f4-a1
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-06-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- just ask to do some shit and know it’ll do it
- very obvious what it could actually do
- not the case right now
- A better ChatGPT would quite literally be a different product
Our interpretation
- Read the execution claim as saying users cannot generally assume successful completion without checking. Do not read it as saying ChatGPT never completes a request.
- Read capability clarity as whether ordinary users can anticipate which requests will succeed, rather than merely whether feature documentation exists.
- 'A different product' is an evaluative description without a stated product-identity test or forecast deadline.
- Use the current calendar year through publication, 2025-01-01 to 2025-06-09, as the reporting window for the ongoing state. Assess versions and features available during that window.
Counts as supported if: Contemporaneous product evaluations show consequential execution failures and poor communication of capability limits; user evidence shows that expected success boundaries are unclear.
Counts as contradicted if: Representative evaluations show dependable execution within clearly communicated capabilities, and user evidence demonstrates that ordinary users can identify those boundaries.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
OpenAI's own documentation supports the concern about unreliable results and poorly communicated uncertainty. It also described concrete capabilities. Those facts do not settle whether ordinary users could anticipate successful execution across ChatGPT as a whole. The assertion that improvement would make it a different product has no measurable criterion. The complete claim therefore remains unresolved.
Limitations
- Evidence about one feature cannot establish the reliability of every ChatGPT model and use.
- Documented features do not establish clear success boundaries.
- Only original launch material is used; later updates on the live pages are excluded.
- The product-identity clause is evaluative and is not independently scored.
Evidence (2 items)
Supports the claim: OpenAI,
uncertainty
Why it matters here: OpenAI acknowledged both source-assessment failures and difficulty communicating uncertainty in ChatGPT's deep research feature. This supports the warning against assuming successful, trustworthy execution, but covers only one feature.
- Where
- Limitations
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: OpenAI,
images
Why it matters here: The launch documentation described concrete operations for models available within ChatGPT. Documentation is material context against total capability obscurity, but it does not show that users understood when those operations would succeed.
- Where
- Thinking with images
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
Contemporaneous tests of representative ChatGPT requests, paired with user studies of expected versus actual success. The product-identity clause would additionally need an operational definition.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Never Forget What They've Done
One that you can just ask to do some shit and know it’ll do it, and it’d also be very obvious what it could actually do, which is not the case right now. A better ChatGPT would quite literally be a different product.
- Where
- paragraph 25
- Published
- 2025-06-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-never-forget-what-theyve-done-1a44f4-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment wyea-never-forget-what-theyve-done-1a44f4-a1.