“They can help a coder learn a new API, or maybe even a new programming language.”
Our reading:
AI coding tools can help a coder learn a new API, and maybe even a new programming language.
Assessment: Supported
What happened
Was it true for the period described?
Outcome: Supported
The evidence matches what they said.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
Marcus makes a modest capability claim. The observational study documents programmers using Copilot to explore unfamiliar APIs, including checking suggested matplotlib arguments against documentation. That supports practical help with learning API usage. Comprehension difficulties in another study limit the reliability of that help, but do not contradict 'can help'. The tentative programming-language extension is not treated as a promise of language mastery.
Caveats
- Support concerns practical assistance, not measured long-term learning gains.
- The direct evidence concerns Copilot and a small observational sample.
- The findings predate the post; they establish earlier capability rather than measuring every tool available in March 2025.
- The counterevidence quotation comes from a reproduction of the original study's abstract.
Evidence (2 items)
Supports the claim: arXiv; Shraddha Barke, Michael B. James, and Nadia Polikarpova,
reading the documentation
Why it matters: The study observed 20 programmers and describes unfamiliar-API exploration. P11 used suggested matplotlib arguments and checked their meaning against documentation; this supports practical assistance in discovering API usage, with human verification.
- Publisher
- arXiv; Shraddha Barke, Michael B. James, and Nadia Polikarpova
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: Machine Learning for Big Code and Naturalness; abstract by Priyan Vaithilingam, Tianyi Zhang, and Elena Glassman,
difficulties in understanding
Why it matters: The original CHI 2022 study documents comprehension difficulties, limiting any inference of reliable or automatic learning. This is contextual counterevidence because it does not establish that assistance with unfamiliar APIs is impossible; the availability date refers to the original conference publication.
- Publisher
- Machine Learning for Big Code and Naturalness; abstract by Priyan Vaithilingam, Tianyi Zhang, and Elena Glassman
- Kind of source
- Secondary summary
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The plain reading is that AI coding tools can provide practical assistance in learning to use an API unfamiliar to the coder. It does not promise mastery, lasting retention, or benefits for every coder.
- The programming-language extension is explicitly tentative. It is not a firm commitment that tools can teach an entire language.
- The claim concerns capability established by the post date, 2025-03-12. No reporting cycle or starting date is supplied. Earlier demonstrations of that capability are admissible; restricting the observation to calendar 2024 would be arbitrary.
Right if: Credible observations or studies show coders using existing AI coding tools to understand or discover how to use unfamiliar APIs, with practical assistance beyond merely receiving unexplained code.
Wrong if: Evidence establishes that the claimed practical assistance was unavailable in tools existing by the post date, or that apparent supporting cases did not involve understanding unfamiliar APIs. Individual failures or an absence of average learning gains would not alone disprove 'can help'.
We wrote this test after the outcome was known.
We found 10 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, AI Coding Fantasy meets Pac-Man, paragraph 9.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-03-12.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, AI Coding Fantasy meets Pac-Man
They can help a coder learn a new API, or maybe even a new programming language.
- Where
- paragraph 9
- Published
- 2025-03-12
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Supported. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.