Gary Marcus Marcus on AI Published 2024-06-01

26 months of ridicule and failure

We recorded 5 claims from this post. 3 were checked against evidence. 2 can’t be checked. Read the post at its source.

Claims from this post (5)

In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.

Assessment: SupportedSaid 2024-06-01
“The ridicule is of me; the failure is of the lords of deep learning to resolve what ails their AI.”
What we tested: The lords of deep learning have failed to resolve what ails their AI.
What happened: The leading lab says its flagship model, released within the period, still hallucinates and is not fully reliable. (OpenAI, 2023-03-14)
Assessment: Can’t be checkedSaid 2024-06-01
“But here’s the thing, what I said then to Bach still holds, 100%, 26 months later”
What we tested: What Marcus said to Joscha Bach 26 months earlier still holds, 100%.
Assessment: UnresolvedSaid 2024-06-01
“Worse, we are no closer to a solution to the alignment problem now than we were then.”
What we tested: There has been no progress toward solving the AI alignment problem between 2022 and mid-2024.
Assessment: SupportedSaid 2024-06-01
“For all the daily claims of “exponential progress”, and for all the tens of billions that have been invested since, reliability is still a dream, and of course toxicity and hallucinations are still with us.”
What we tested: Despite tens of billions of dollars invested since 2022, deep learning systems are still unreliable and still produce toxicity and hallucinations.
What happened: The May evaluation documents invented constraints and false claims of task completion, directly supporting continuing hallucinations and unreliable autonomous execution. The report also records successful shorter tasks and strong scientific-question performance, so the failures do not imply universal incompetence. (UK AI Safety Institute, 2024-05)
Assessment: Can’t be checkedSaid 2024-06-01
“For not so good measure, one of the few people who recognized, early, the potential dangers of unaligned AGI, Musk, has switched teams, flipping from calling for a pause to going all in on—and wildly overhyping—a technology that remains exactly as incorrigible as it ever was.”
What we tested: Current AI technology, which Musk is now wildly overhyping, remains exactly as incorrigible as it ever was.

Claims from this post (5)

“The ridicule is of me; the failure is of the lords of deep learning to resolve what ails their AI.”

“But here’s the thing, what I said then to Bach still holds, 100%, 26 months later”

“Worse, we are no closer to a solution to the alignment problem now than we were then.”

“For all the daily claims of “exponential progress”, and for all the tens of billions that have been invested since, reliability is still a dream, and of course toxicity and hallucinations are still with us.”

“For not so good measure, one of the few people who recognized, early, the potential dangers of unaligned AGI, Musk, has switched teams, flipping from calling for a pause to going all in on—and wildly overhyping—a technology that remains exactly as incorrigible as it ever was.”