“I think it’s fair to say that the multimodal stuff still is very buggy, having troubles with basics like the English alphabet.”
Our reading, the sentence that was checked:
OpenAI's multimodal capabilities remain very buggy as of April 2024, struggling even with basics like the English alphabet.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, TED, AI, and the test of time, paragraph 3.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-ted-ai-and-the-test-of-time-a94e33-a1
We recorded 11 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2024-04-15. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- the multimodal stuff still is very buggy
- having troubles with basics like the English alphabet
Our interpretation
- The context identifies OpenAI's multimodal products, especially image generation accessed through ChatGPT.
- The plain reading is recurring failures on elementary tasks, including rendering letters correctly. It does not require failure on every request.
- For the ongoing state, use the latest completed calendar quarter and the subsequent period before publication: January 1 through April 15, 2024. This reporting window is an evaluator choice.
- A single failed image establishes a possible failure, but does not by itself establish the broader description 'very buggy'.
Counts as supported if: Contemporaneous tests of the deployed products establish recurring elementary failures, including letter or alphabet rendering.
Counts as contradicted if: Representative contemporaneous tests establish dependable performance on the elementary multimodal tasks described, including letter and alphabet rendering.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The readable benchmark establishes limitations in image generation, alongside strong comparative performance by DALL-E 3. It does not test the alphabet or establish recurring elementary failures. Relevant contemporaneous reporting could not be verified through the required reader. This leaves the claim unsettled, rather than disproved.
Limitations
- The assessment is limited by source-access failures.
- Advanced-task failures cannot establish the specific elementary-task claim.
- The severity implied by 'very buggy' remains qualitative.
Evidence (1 item)
Context only: Zhiqiu Lin and colleagues, arXiv,
rated significantly lower
Why it matters here: The evaluation includes DALL-E 3 and finds weaknesses on advanced compositions. It also ranks DALL-E 3 above the other tested models on nearly all skills; neither finding settles recurring elementary alphabet failures.
- Publisher
- Zhiqiu Lin and colleagues, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Dated, reproducible tests of the OpenAI products deployed before April 15, 2024, with prompts and repeated outputs for alphabet rendering and other elementary multimodal tasks.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, TED, AI, and the test of time
I think it’s fair to say that the multimodal stuff still is very buggy, having troubles with basics like the English alphabet.
- Where
- paragraph 3
- Published
- 2024-04-15
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-ted-ai-and-the-test-of-time-a94e33-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-ted-ai-and-the-test-of-time-a94e33-a1.