“First, AI can’t handle one of the most essential tasks of software engineering, which is collaboratively discussing the requirements of a project and how to implement the technology.”
Our reading, the sentence that was checked:
AI cannot collaboratively discuss project requirements and implementation, an essential software engineering task.
Assessment: Unresolved Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, They're Looting The Internet, paragraph 48.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-the-great-looting-of-the-internet-318900-a2
We recorded 21 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-04-15. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- AI can’t handle
- collaboratively discussing the requirements of a project
- how to implement the technology
Our interpretation
- The ordinary reading is that then-current AI could not substantively exchange questions, clarify software requirements, and discuss implementation choices with a person.
- Successful collaboration need not mean autonomous delivery or flawless performance.
- Although the surrounding discussion concerns Devin, the sentence expressly refers to AI generally.
- No reporting period is specified. I use 2024 through publication as the current-state observation window; earlier experiments provide context.
Counts as supported if: Direct evaluations of then-current systems show that they could not conduct substantive, responsive discussions covering project requirements and implementation.
Counts as contradicted if: A documented interaction with a then-current system shows substantive clarification of software requirements and discussion of implementation choices with a person.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Cognition's March 12, 2024 announcement describes Devin accepting feedback and discussing design choices, but the supplied text contains no complete exchange demonstrating requirements clarification and implementation discussion [e1]. The study submitted July 14, 2023 evaluates generated requirements, a narrower task outside the January 1–April 15, 2024 assessment period [e2]. These items establish neither successful performance of the full collaborative task during that period nor a general inability to perform it [e1, e2].
Limitations
- The vendor description does not verify the capability through an inspectable interaction [e1].
- The earlier requirements study does not test the full collaborative task during the assessment period [e2].
Evidence (2 items)
Context only: Cognition,
design choices
Why it matters here: Cognition expressly described interactive design work. This challenges the categorical wording, but the retrieved text does not supply a complete exchange demonstrating requirements clarification and implementation discussion.
- Publisher
- Cognition
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: arXiv,
six questions
Why it matters here: The researchers evaluated AI-generated requirements against expert responses. This is relevant earlier evidence of assistance, but neither its period nor its task directly matches the full claim.
- Where
- Abstract
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A complete human–AI exchange from January 1–April 15, 2024 showing substantive requirements clarification and implementation discussion would contradict the claim. Direct evaluations of then-current systems establishing the claimed general inability would support it.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, They're Looting The Internet
First, AI can’t handle one of the most essential tasks of software engineering, which is collaboratively discussing the requirements of a project and how to implement the technology.
- Where
- paragraph 48
- Published
- 2024-04-15
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-the-great-looting-of-the-internet-318900-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment wyea-the-great-looting-of-the-internet-318900-a2.
HeldBy this site
The final-text check failed: rationale_supported_by_evidence; overclaims: Vendor descriptions cannot alone settle this disputed capability, and failure on one Upwork task cannot establish that AI generally lacked it..
About assessment wyea-the-great-looting-of-the-internet-318900-a1, which was not published.