“Not sure that’s a great idea, in an era in which AI still can’t quite sort out what’s happening with an elephant on a crowded beach.”
Our reading:
AI still can't quite sort out what's happening with an elephant on a crowded beach.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The accessible records do not preserve enough of the referenced elephant demonstration to verify its prompt, output, and failure. The search included the October example rather than requiring a January experiment. A third-party claim of improvement uses a changed prompt and cannot establish that Marcus misdescribed the original result. Neither support nor contradiction is established.
Caveats
- Plain-text extraction did not recover the crucial visual records.
- The original demonstrator and exact task remain uncertain.
- Discussion of altered prompts is not a replication of the referenced task.
Evidence (1 item)
Context only: Fabio, third-party commenter on Substack,
Given a slightly different prompt, the results are way better.
Why it matters: A commenter reports better results after changing the prompt. This is a relevant counterclaim, but neither verifies the original demonstration nor establishes that the revised output satisfied the original requirements.
- Publisher
- Fabio, third-party commenter on Substack
- Kind of source
- Secondary summary
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- can't quite (their words)
Where we had to interpret
- Assess the factual reference to the elephant example, not the preceding opinion about whether military deployment is a good idea.
- Read 'can't quite' as a demonstrated difficulty meeting or interpreting the referenced scene's requirements. It does not assert that every AI system fails every elephant-related task.
- The context includes a new Colin Fraser demonstration and an October newsletter containing a Dave Andress example. The final sentence's exact referent must be established from the original records rather than assumed.
- The target period runs from the beginning of the explicitly referenced October through publication. October records are admissible; no new January experiment is required.
Right if: An independently accessible original or authenticated record identifies the referenced elephant task and shows that its AI output materially failed the prompt or misinterpreted the scene.
Wrong if: An authenticated record of the referenced task shows that the output met the relevant requirements, or establishes that the alleged failure was misdescribed or attributed to the wrong system.
We wrote this test after the outcome was known.
What would settle it
An original or authenticated record, available by the evaluation date, preserving the referenced elephant prompt, image, response, system identity, and date. The October record is eligible if it is the example Marcus meant.
We found 11 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Where’s Waldo? The Elephant in the Room, paragraph 12.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-29. The post may have been edited after 2024-01-25.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Where’s Waldo? The Elephant in the Room
Not sure that’s a great idea, in an era in which AI still can’t quite sort out what’s happening with an elephant on a crowded beach.
- Where
- paragraph 12
- Published
- 2024-01-25
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.