BREAKING NEWS: AI coding may not be helping as much as you think
We recorded 6 claims from this post. 2 were checked against evidence. 4 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (6)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Observation
“A significant fraction of the money that people have paid for Generative AI has been for coding assistance.”
Our reading: A significant fraction of the money people have paid for generative AI has been for coding assistance.
No horizon Evidence as of
Limitation: The evidence supplies no matched coding-payment numerator and total generative AI-payment denominator covering the same period and market.
Assessment: UnresolvedSaid
Observation
“When I point to the positive uses of Generative AI, I invariably point to how GenAI serves as kind of valuable autocomplete for coding.”
Our reading: When pointing to positive uses of generative AI, Marcus invariably points to how it serves as kind of a valuable autocomplete for coding.
No horizon Evidence as of
Limitation: Search coverage is incomplete and some interviews are edited.
Assessment: Can’t be checkedSaid
Observation
“People might be imagining productivity gains that they are not getting, and ignoring real-world costs, to boot.”
Our reading: People using AI coding tools may be imagining productivity gains they are not getting while ignoring real-world costs.
No horizon
Qualifier: if the METR finding is general and replicable
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“IF this is a general, replicable finding, it’s a serious blow to generative AI’s flagship use case.”
Our reading: If METR's finding that AI tools slowed down task completion is general and replicable, it is a serious blow to generative AI's flagship use case.
No horizon
Only if: if this is a general, replicable finding
Assessment: Can’t be checkedSaid
Causal claim
“Debugging code you didn’t write can be hard.”
Our reading: One possible mechanism behind the AI-coding slowdown is that debugging code you didn't write (AI-generated code) can be hard.
No horizon
Qualifier: if the METR finding is true
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But for now, the disconnnect between what coders thought they would get out of the tools efficiency-wise and what they actually did get out of them is cause for reevaluation.”
Our reading: The gap between coders' expected and actual efficiency gains from AI tools warrants reevaluating the tools' value.
No horizon
Qualifier: for now
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.