“(And it is not even clear that LLMs do this any better at this than the many other older techniques studied in 2017.)”
Our reading:
It is not clear that LLMs infer geography from text any better than older techniques studied in 2017.
Assessment: Can’t be checked
What happened
Not graded on evidence
Outcome: Can’t be checked
An opinion, a recommendation, or too vague to test. Listed, not scored.
Evidence up to . Checked .
Why
Marcus says superiority is “not even clear.” He does not assert equal or inferior performance, nor explicitly claim that no published comparison shows an advantage. Scoring either proposition would strengthen his words. The rule allowing an ordinary measure for firm claims does not apply to this qualified uncertainty. The claim should therefore remain unscored.
Caveats
- This decision does not establish whether LLMs outperformed older methods.
- The research review was limited to context; no exhaustive absence-of-evidence claim is made.
Evidence (1 item)
Claims like this aren’t scored, so there is no evidence to show.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- In context, “this” means recovering geographic information from text.
- Marcus expresses uncertainty about comparative performance as of the post date. He does not firmly assert that LLMs perform no better or that no published comparison demonstrates an advantage.
- Treating this as a literature-wide claim about the absence of superior results would add a commitment his words do not supply.
We don’t grade claims like this. We only check the quote and who said it.
What would settle it
A separate, explicit claim identifying the comparative result or evidence standard could be assessed. It should not be retroactively substituted for this wording.
We found 10 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Muddles about Models, paragraph 7.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-29. The post may have been edited after 2023-10-05.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Muddles about Models
(And it is not even clear that LLMs do this any better at this than the many other older techniques studied in 2017.)
- Where
- paragraph 7
- Published
- 2023-10-05
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Can’t be checked. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.