“Put simply, OpenAI is testing for something that doesn’t actually represent the majority of ChatGPT usage.”
Our reading:
What OpenAI tests for with SimpleQA does not represent the majority of ChatGPT usage.
Assessment: Supported
What happened
Was it true for the period described?
Outcome: Supported
The evidence matches what they said.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
SimpleQA measures only short, fact-seeking questions with a single, verifiable answer (e1). An OpenAI and Harvard working paper looked at a representative sample of consumer ChatGPT conversations from May 2024 to June 2025. The post date falls inside that span (e2). In that study, the 'Seeking Information' topic was 14% of conversations in June 2024 and 24% in June 2025 (e2, e3). This category is broader than SimpleQA-style lookups. By mid-2025, Practical Guidance was about 29% and Writing was 24% (e2). The 'Asking' category was 49% of messages (e4). It is wider still and includes advice. Because the broader information-seeking category stayed well under half of usage in this period, SimpleQA-style lookups were not a majority of ChatGPT usage. That meets the contract's support condition.
Caveats
- The topic shares come from a news report on the working paper (e2, e3), because the full paper could not be fetched.
- The study covers consumer plans only, not business or API use.
- No source directly counts SimpleQA-style single-fact questions. The 'Seeking Information' share serves only as an upper bound.
- The working paper is not peer-reviewed.
Evidence (4 items)
Supports the claim: Search Engine Journal,
Practical Guidance remains around 29%. Writing declined from 36% to 24% over the past year. Seeking Information grew from 14% to 24%.
Why it matters: By mid-2025, information-seeking, a broader category than SimpleQA lookups, was about a quarter of usage, behind practical guidance and level with writing.
- Publisher
- Search Engine Journal
- Kind of source
- News report, unaudited
- About events on
- Published
Supports the claim: Search Engine Journal,
“Seeking Information” messages now account for 24% of ChatGPT conversations, up from 14% a year earlier.
Why it matters: About nine months before the post, information-seeking was only 14% of conversations. Factual lookups were a small minority throughout the period.
- Where
- paragraph 1
- Publisher
- Search Engine Journal
- Kind of source
- News report, unaudited
- About events on
- Published
Supports the claim: OpenAI,
About half of messages (49%) are “Asking,” a growing and highly rated category
Why it matters: Even 'Asking', a wide category that includes advice as well as facts, is just under half of messages. Short factual lookups are therefore a smaller share still.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding, unaudited
- About events on
- Published
Context only: OpenAI,
it only measures factuality under the constrained setting of short, fact-seeking queries with a single, verifiable answer.
Why it matters: Shows what OpenAI tests with SimpleQA: short factual queries with a single answer.
- Where
- section Conclusions
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- SimpleQA (their words)
Where we had to interpret
- What SimpleQA tests is short, fact-seeking questions with a single checkable answer.
- Plain reading: short factual lookups of this kind are less than half of ChatGPT usage, measured by share of messages or conversations.
- Target period: the year around the post, from mid-2024 to mid-2025. Zitron describes an ongoing state without naming a period. The only representative usage data covers this span, and the post falls inside it.
Right if: Representative ChatGPT usage data from the target period shows that information-seeking, a category broader than SimpleQA-style lookups, is well under half of usage.
Wrong if: Representative usage data from the target period shows that short factual lookups make up a majority of ChatGPT messages or conversations.
We wrote this test after the outcome was known.
We found 26 claims in this post. See them all.
Full record
Source and dates
Where's Your Ed At, The Phony Comforts of AI Optimism, paragraph 21.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-03-24.
Where it was said (1)
Said once in the posts we read.
: Where's Your Ed At, The Phony Comforts of AI Optimism
Put simply, OpenAI is testing for something that doesn’t actually represent the majority of ChatGPT usage.
- Where
- paragraph 21
- Published
- 2025-03-24
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Supported. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.