Grok 3 Beta in Shambles
We recorded 1 claim from this post. 1 can’t be checked. 8 more are held. Read the post at its source.
Claims from this post (1)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But today’s date should never be multiple guess.”
Our reading: An AI system like Grok 3 should give today's date consistently, not different answers on different tries.
No horizon
Held from this post (8)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: In the author's own testing, Grok 3 Beta answered the first question correctly and then made many errors.
- The criteria are ambiguous. Our reading, not assessed: Grok 3's Deep Search makes amazing-seeming reports that seem, reasonably often, to contain very subtle and difficult-to-detect errors.
- The record does not meet the current rules. Our reading, not assessed: AI valuations just keep rising regardless of the results, at least for now.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: As in the early ChatGPT days, LLM output often contains subtle errors most people would not catch, however good it looks, so users should beware.
- The two reviews reached different assessments. Our reading, not assessed: Grok 3 required training on two new massive data centers operating full time for months and 15x the compute of Grok 2, yet all these kinds of errors feel awfully familiar to the author.
- The record does not meet the current rules. Our reading, not assessed: When the author asked, perhaps too tersely, 'what happened to Billings, Montana?', Grok (Deep Search) did not understand he meant the omission from its table and instead described a recent earthquake there, which from his fact-checking does not appear to have happened.