“Regardless of how well it works (it works very very poorly and inconsistently), it's also likely very expensive.”
Our reading:
OpenAI's ChatGPT agent works very poorly and inconsistently and is probably very expensive to run.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
Independent launch testing supports the narrower concern that the agent was slow, glitchy and sometimes unreliable. OpenAI's benchmark results also show useful capabilities. These records do not settle the broad claim that it worked very poorly and inconsistently across ordinary tasks. The explicitly hedged expense statement is not turned into a firm cost claim or used to determine the outcome.
Caveats
- The independent review covers a small task sample.
- Company benchmarks are not representative independent reliability measurements.
- The cost clause remains an ungraded estimate.
Evidence (2 items)
Supports the claim: The Verge,
not always reliable
Why it matters: Hands-on testing supports concerns about launch reliability. Its limited task sample does not establish how frequently the product fails across ordinary use; the shopping example also involved separate browser sessions, and banking transfers were restricted tasks.
- Publisher
- The Verge
- Kind of source
- Independent measurement
- About events on
- Published
Context only: OpenAI,
68.9%
Why it matters: The launch disclosure reports substantial capability on a difficult browsing benchmark, qualifying a blanket description of poor performance. A benchmark score does not establish dependable completion of ordinary tasks or consistency across repeated attempts.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The firm assertion concerns the launch version of ChatGPT agent, from its July 17 announcement through the July 21 post.
- The ordinary performance reading is frequent substantive failures and unreliable completion of ordinary tasks the product was advertised to handle.
- No numerical success threshold is supplied, so none is invented.
- The expense statement is explicitly hedged and supplies no cost measure or comparison. It is recorded but does not determine the performance assessment.
Right if: Representative evaluations of the launch version show frequent substantive failures and unreliable completion across ordinary advertised tasks.
Wrong if: Representative evaluations of the launch version show broadly dependable completion across ordinary advertised tasks, inconsistent with the description of very poor and inconsistent performance.
We wrote this test after the outcome was known.
What would settle it
Independent evaluations of the July 2025 launch version across representative advertised tasks, including repeated attempts and documented completion criteria.
We found 56 claims in this post. See them all.
Full record
Source and dates
Where's Your Ed At, The Hater's Guide To The AI Bubble, paragraph 244.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-07-21.
Where it was said (1)
Said once in the posts we read.
: Where's Your Ed At, The Hater's Guide To The AI Bubble
Regardless of how well it works (it works very very poorly and inconsistently), it's also likely very expensive.
- Where
- paragraph 244
- Published
- 2025-07-21
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.