“It got my first question right (a comparison between two decimals, 56.1012 vs 56.90) but things went rapidly downhill from there.”
Our reading:
In the author's own testing, Grok 3 Beta answered the first question correctly and then made many errors.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The claim describes one testing session and does not assert that most subsequent answers were wrong. Searches found Marcus's account and other people's tests, but no independently verifiable record establishing both the correct first response and the repeated later errors. The available evidence therefore does not settle the account.
Caveats
- The screenshots could not be assessed from the returned plain text.
- Failure to locate a session record is not evidence that the account is false.
- Other users' successes or failures cannot establish this session's sequence.
Evidence (0 items)
Nothing we found settles this yet.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- author's own testing over a couple of hours (their words)
Where we had to interpret
- The claim concerns Marcus's particular testing session, not Grok 3's average performance.
- The plain reading is that the first decimal comparison was correct and multiple distinct errors followed during the session. It does not require errors on most subsequent prompts.
- The session is assigned to the post date, 2025-02-19, based on 'I just took it for a spin.' Its precise timestamps are not supplied.
- An authenticated record showing the responses and their order could establish the account without a statistical benchmark or an error-rate threshold.
Right if: A verifiable record of Marcus's session shows that Grok correctly answered the initial decimal comparison and subsequently made multiple distinct, demonstrable errors consistent with the described sequence.
Wrong if: A verifiable session record shows that the initial comparison was answered incorrectly, or a sufficiently complete record shows that the claimed repeated subsequent errors did not occur. Successful answers from other sessions do not contradict this account.
We wrote this test after the outcome was known.
What would settle it
An authenticated session export or independently preserved transcript showing the first decimal comparison and the subsequent exchanges, with enough context to check the alleged errors.
We found 3 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Grok 3 Beta in Shambles, paragraph 2.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-02-19.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Grok 3 Beta in Shambles
It got my first question right (a comparison between two decimals, 56.1012 vs 56.90) but things went rapidly downhill from there.
- Where
- paragraph 2
- Published
- 2025-02-19
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.