“Too many edge cases and not enough reasoning.”
Our reading:
Current deep-learning AI systems face too many edge cases and have too little reasoning ability.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
There is evidence behind the concern: Microsoft acknowledged practical failures, and a contemporary ChatGPT study found unreliable reasoning. Another study also found meaningful reasoning strengths. These records support a narrower observation of uneven performance, but do not settle whether the systems discussed had insufficient reasoning and edge-case robustness for their intended uses. ChatGPT benchmark results cannot simply be assigned to Bing, Bard, or FSD.
Caveats
- The outcome depends on anchoring the sentence to the products discussed immediately before it.
- A broader reading that merely asserts the existence of contemporary robustness and reasoning weaknesses would be easier to support.
- The research evidence consists of preprints and does not supply a common real-world adequacy standard.
Evidence (3 items)
Supports the claim: Microsoft,
challenges
Why it matters: Microsoft acknowledged practical answer-quality weaknesses and also described problems in extended conversations. This supports the reliability concern, but does not establish inadequate reasoning or its prevalence across the named systems.
- Publisher
- Microsoft
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
Context only: Yejin Bang and colleagues, arXiv,
they are not reliable.
Why it matters: The original study reports substantial variation and weaknesses in ChatGPT's reasoning. It provides context for the broader criticism, but tests a different system and does not establish test dates within the selected observation window.
- Publisher
- Yejin Bang and colleagues, arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: Chengwei Qin and colleagues, arXiv,
arithmetic reasoning tasks
Why it matters: This study found reasoning strengths as well as weaknesses, cautioning against a blanket reading of the criticism. Its comparisons concern ChatGPT and benchmark tasks, not whether the named products met their intended-use requirements.
- Publisher
- Chengwei Qin and colleagues, arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- Read the sentence in its immediate context as describing practical weaknesses of the contemporary systems exemplified by Tesla FSD, Bing, and Bard.
- The ordinary reading is that unusual inputs or situations exposed material reliability problems and that reasoning was insufficient for dependable intended use. It does not assert that the systems had no reasoning ability.
- Use the latest reporting window identified by the surrounding post: the preceding calendar week through February 16, 2023. This is a snapshot of contemporary performance, not a claim about later systems.
- Evaluations of other contemporary systems can inform the broader observation, but cannot by themselves establish the capabilities of the three named products.
Right if: Contemporaneous evaluations or failure analyses establish both material failures on unusual situations and inadequate reasoning for the intended uses of the systems under discussion.
Wrong if: Representative evaluations of the relevant systems establish dependable handling of unusual situations and sufficient reasoning for their intended uses, undermining the claimed practical shortfall.
We wrote this test after the outcome was known.
What would settle it
Dated evaluations of the relevant February 2023 versions that test unusual situations and reasoning against their intended tasks, with representative failure rates and documented consequences.
We found 3 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Smells a little bit like AI winter?, paragraph 9.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-29. The post may have been edited after 2023-02-16.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Smells a little bit like AI winter?
Too many edge cases and not enough reasoning.
- Where
- paragraph 9
- Published
- 2023-02-16
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.