“And, as I noted at the time, it was not particularly reliable, failing to accurately count the number of times the letter ‘r’ appeared in the word “strawberry.””
Our reading, the sentence that was checked:
OpenAI's o1 was not particularly reliable at launch, failing to count the r's in 'strawberry'.
Assessment: Unresolved Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Deep Impact, paragraph 16.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-deep-impact-a1a8da-a1
We recorded 50 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-01-29. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- at launch (qualifier, stated by the speaker)
From the speaker
- not particularly reliable
- failing to accurately count
- the letter ‘r’
- “strawberry”
- launch its o1 reasoning model last year
Our interpretation
- The launch reference means the September 2024 o1-preview release, rather than later o1 versions.
- The ordinary reading is that launch-version o1 could fail this simple counting task. It does not say every attempt failed.
- The broader reliability judgment requires more than one selected failure. I assess that separately from whether a strawberry failure occurred.
- The target period is the September launch and its initial public testing.
Counts as supported if: A version-identifiable launch-period transcript or reproducible test establishes the strawberry failure, with broader testing supporting the characterization of unreliable performance.
Counts as contradicted if: Evidence establishes that the alleged failure involved another model or was misreported, and launch-version testing contradicts the reliability characterization. A successful strawberry answer alone does not disprove occasional failures.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Contemporaneous reports did allege strawberry failures, so the example was not invented retrospectively. However, the linked source does not expose the failed exchange or test settings. Other material reports correct answers. That does not disprove an occasional failure, but the retrieved records do not securely establish the specific failure and the broader reliability judgment together.
Limitations
- The original failure may survive in an image or conversation that read_page did not expose.
- Selected successes and failures cannot establish an overall failure rate.
Evidence (3 items)
Context only: Futurism,
Tasks as basic as counting
Why it matters here: The article reports several launch-period errors and separately attributes the strawberry allegation to users. It supplies neither a representative reliability measurement nor a complete strawberry failure transcript.
- Publisher
- Futurism
- Kind of source
- News report
- Describes events of
- Public from
Context only: Navid Matin Moghaddam on X,
they were successful
Why it matters here: This is the source linked by Futurism for the strawberry allegation. Its readable text repeats others' results without identifying their model settings or showing the failed exchange.
- Publisher
- Navid Matin Moghaddam on X
- Kind of source
- Secondary summary
- Describes events of
- Public from
Context only: 16x Prompt,
accurately counting letters
Why it matters here: The page explicitly describes correct strawberry answers and labels the relevant update September 12. Correct answers limit any reading of universal inability, but do not exclude occasional failures; the available page was subsequently updated.
- Publisher
- 16x Prompt
- Kind of source
- Secondary summary
- Describes events of
- Public from
What would settle it
A preserved September 2024 o1-preview failure transcript, plus repeated or representative launch-version testing sufficient to assess the broader reliability claim.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Deep Impact
And, as I noted at the time, it was not particularly reliable, failing to accurately count the number of times the letter ‘r’ appeared in the word “strawberry.”
- Where
- paragraph 16
- Published
- 2025-01-29
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-deep-impact-a1a8da-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.