The mirage of visual understanding in current frontier models
We recorded 5 claims from this post. 2 were checked against evidence. 3 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (5)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Observation
“AGI this stuff ain’t.”
Our reading: The frontier models described in the study are not AGI.
No horizon Evidence as of
Limitation: The result depends on how broadly competence must extend and what shortfalls remain compatible with AGI.
Assessment: SupportedSaid
Observation
“This study reinforces what Anh Totti Nguyen has been saying for a long time, in a series of underappreciated papers like Vision Language Models are Blind that I keep trying to draw attention to.”
Our reading: The new Stanford study reinforces what Anh Totti Nguyen has long been saying in a series of underappreciated papers, such as 'Vision Language Models are Blind'.
No horizon Evidence as of
Limitation: The studies test different model versions, tasks, and failure mechanisms.
Assessment: Can’t be checkedSaid
Forecast
“Also, re the very active discussion on AI and jobs: although some white collar jobs (e.g., entry-level coder or market research assistant) may be in near-term jeopardy, many of those that require visual understanding (architect, cartographer, civil engineer, film editor, medical illustrator, urban planner, etc) probably aren’t vulnerable until entirely new techniques are developed.”
Our reading: Some white collar jobs, such as entry-level coder or market research assistant, may be in jeopardy from AI in the near term.
No horizon
Qualifier: e.g., entry-level coder or market research assistant
Assessment: Can’t be checkedSaid
Conditional forecast
“Also, re the very active discussion on AI and jobs: although some white collar jobs (e.g., entry-level coder or market research assistant) may be in near-term jeopardy, many of those that require visual understanding (architect, cartographer, civil engineer, film editor, medical illustrator, urban planner, etc) probably aren’t vulnerable until entirely new techniques are developed.”
Our reading: Many jobs requiring visual understanding (architect, cartographer, civil engineer, film editor, medical illustrator, urban planner) probably won't be vulnerable to AI until entirely new techniques are developed.
No horizon
Only if: until entirely new techniques are developed
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If your humanoid robot can’t understand the visual world, it’s just a demo, and not something you can trust.”
Our reading: A humanoid robot that can't understand the visual world is just a demo and not something you can trust.
No horizon
Scope: humanoid robots that can't understand the visual world
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.