“The most interesting part was playing with Deep Search which makes amazing-seeming reports that seem reasonably often to have very subtle, and difficult to detect errors.”
Our reading:
Grok 3's Deep Search makes amazing-seeming reports that seem, reasonably often, to contain very subtle and difficult-to-detect errors.
Assessment: Can’t be checked
What happened
Not graded on evidence
Outcome: Can’t be checked
An opinion, a recommendation, or too vague to test. Listed, not scored.
Evidence up to . Checked .
Why
Marcus says the reports 'seem reasonably often' to contain subtle errors. This is a hedged impression with no defined frequency or detection standard. Turning it into a scored frequency claim would strengthen his words. Independent testing can assess particular errors, but cannot fairly settle this sentence as written.
Caveats
- This decision does not establish that Deep Search reports were accurate or inaccurate.
- The contextual review covers a different sample and does not measure how difficult errors were to detect.
Evidence (1 item)
Claims like this aren’t scored, so there is no evidence to show.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- seem to have (their words)
- reasonably often (their words)
Where we had to interpret
- The sentence expresses a hedged impression rather than a firm, measurable frequency claim.
- Neither a numerical meaning for 'reasonably often' nor a reader population for 'difficult to detect' is supplied.
- Assigning an error rate or detection threshold would strengthen the speaker's commitment. The plain-reading allowance for firm language does not apply.
We don’t grade claims like this. We only check the quote and who said it.
What would settle it
A separate, firm claim specifying the reports covered, the frequency asserted, and what makes an error difficult to detect would permit assessment.
We found 3 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Grok 3 Beta in Shambles, paragraph 8.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-02-19.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Grok 3 Beta in Shambles
The most interesting part was playing with Deep Search which makes amazing-seeming reports that seem reasonably often to have very subtle, and difficult to detect errors.
- Where
- paragraph 8
- Published
- 2025-02-19
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Can’t be checked. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.