“But you and I both know that they continue to be a major problem that as yet has no known robust solution.”
Our reading, the sentence that was checked:
Outliers remain a major AI problem with no known robust solution.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Dear Elon Musk, here are five things you might want to consider about AGI, paragraph 17.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-dear-elon-musk-here-are-five-things-ffdca9-a2
We recorded 21 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2022-05-31. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- they continue to be a major problem
- as yet
- no known robust solution
Our interpretation
- They refers to outliers and unexpected circumstances discussed immediately before this sentence.
- The ordinary reading concerns reliable handling of unfamiliar inputs and distribution shifts across applications. It does not deny effective methods for particular cases.
- Because academic research has no fixed reporting quarter, the observation is assessed against the current 2022 research record through May 31. Earlier research supplies background.
- A robust solution means broadly demonstrated reliability, not perfect performance on every mathematically possible input.
Counts as supported if: Contemporaneous research documents substantial failures under unexpected inputs or distribution shifts, with available remedies remaining limited rather than constituting a broadly validated solution.
Counts as contradicted if: Research available by the post establishes a known method that reliably resolves these failures across representative applications and unfamiliar conditions.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The February 16, 2022 survey identifies continuing obstacles to handling distribution shifts in graph data, including differences in graph structure, tasks and shift types [e1]. The February 21, 2022 experiments find that fine-tuning can worsen accuracy under distribution shift across ten tested datasets, while a two-stage training method improves those results [e2]. These findings support the observation, as of May 31, 2022, that unfamiliar inputs remained a substantial unresolved problem: they document obstacles and useful remedies without establishing a broadly reliable solution [e1; e2].
Limitations
- The absence of a broadly validated solution is an inference from the inspected public research record, not proof that every possible solution was absent.
- The graph survey and selected distribution-shift datasets do not cover every application or kind of outlier.
- The evidence does not establish that outlier problems are inherently unsolvable or validate every surrounding criticism of deep learning.
Evidence (3 items)
Supports the claim: arXiv; Haoyang Li and colleagues,
highly non-trivial
Why it matters here: The survey identifies unresolved obstacles involving graph structure, differing tasks, and different kinds of distribution shift. It also describes progress, supporting the distinction between useful remedies and a general solution.
- Publisher
- arXiv; Haoyang Li and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv; Ananya Kumar and colleagues,
tradeoff
Why it matters here: Experiments across ten distribution-shift datasets show that improving ordinary accuracy can reduce accuracy after a shift. The proposed two-stage training method improves results, but the paper does not establish a general solution to unexpected inputs.
- Where
- Abstract
- Publisher
- arXiv; Ananya Kumar and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Jingkang Yang and colleagues,
open challenges
Why it matters here: This earlier survey covers anomaly, novelty, open-set, outlier, and out-of-distribution detection. It supplies background on the breadth of the problem and the existence of many partial approaches.
- Publisher
- arXiv; Jingkang Yang and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Dear Elon Musk, here are five things you might want to consider about AGI
But you and I both know that they continue to be a major problem that as yet has no known robust solution.
- Where
- paragraph 17
- Published
- 2022-05-31
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-dear-elon-musk-here-are-five-things-ffdca9-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-dear-elon-musk-here-are-five-things-ffdca9-a2.