Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

“In simpler terms, that’s about 19 hours of extra work per engineer that’s literally just looking at extra code written by LLMs.”

AI Is Too Expensive, paragraph 115.

Our reading:

LLM-generated code at Zillow adds about 19 hours of extra monthly review work per engineer.

Assessment: Unresolved

What happened

Was it true for the period described?

Outcome: Unresolved

We checked, and the evidence so far doesn’t settle it.

Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.

Evidence up to . Checked .

Why

The claim's arithmetic follows from its inputs: 29,000 hours a month divided by about 1,500 engineers is about 19 hours each. Both inputs appear only in the speaker's post, which the contract does not allow as evidence. He attributes the 29,000-hour figure to an internal Zillow deck that is not in the evidence. He does not say where the headcount comes from. The evidence packet contains no Zillow disclosure or independent report that confirms the reviewer-load figure, the headcount, or that the added load comes from reviewing AI-written code. Zillow's June 25, 2026 post says humans review code and test results before anything merges (e1). It gives no review hours or headcount. The evidence neither confirms nor refutes the claim.

Caveats

  • The only source for the key figures is the speaker's post, which cannot be used as evidence.
  • Zillow's post (e1) is dated after the January–May 2026 target period and contains no figures.
  • The period and method behind the reviewer-load figure are not known.
  • Lack of corroboration does not show the figures are wrong.

Evidence (1 item)

  • Context only: Zillow,

    Humans still review code and test results before anything merges. That guardrail is not moving.

    Why it matters: Confirms that Zillow engineers review AI-generated code before it merges. It gives no review-hour totals or engineering headcount, so it cannot test the 19-hour figure.

    Publisher
    Zillow
    Kind of source
    Company disclosure: the company describing itself, not an independent finding
    About events on
    Published

How we judged it

Kind of claim
Observation
Deadline
None given. We don’t add one.

Where we had to interpret

  • Plain reading: Zillow's internal measure of added reviewer load (about 29,000 hours a month), averaged over its engineering headcount (about 1,500), comes to about 19 extra hours of review work per engineer per month, and that added load comes from reviewing AI-generated output.
  • The month or period the deck measured is not stated. The period is set to January 1 through May 19, 2026 (the months before the post) and flagged. It is not limited to April.

Right if: Zillow's deck or another independent source shows added reviewer load of roughly 29,000 hours a month and an engineering headcount of about 1,500, about 19 hours per engineer, tied to AI-generated code.

Wrong if: Zillow's own figures show a materially different added review load or engineering headcount, or show that the added load is not tied to AI-generated output.

We wrote this test after the outcome was known.

What would settle it

Publication of the Zillow deck, a Zillow statement on reviewer-load hours and engineering headcount, or independent reporting by an outlet that has seen the deck.

We found 44 claims in this post. See them all.

Full record

Source and dates

Where's Your Ed At, AI Is Too Expensive, paragraph 115.

Said
We read the source
live page, full text, public
Evidence up to
Checked

We read this wording on 2026-10-28. The post may have been edited after 2026-05-19.

Where it was said (1)

Said once in the posts we read.

  • : Where's Your Ed At, AI Is Too Expensive

    In simpler terms, that’s about 19 hours of extra work per engineer that’s literally just looking at extra code written by LLMs.

    Published
    2026-05-19
    We read it
    , live page
    Access
    full text, public

History

Versions

  • Current: Unresolved. Evidence up to 2026-10-28.

What happened to this record

  1. PublishedBy us

    Published after both AI reviews and the final check passed.

  2. Held backBy us

    A reviewer said our test wasn’t fair to what was actually said.