Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Gary Marcus Causal claim First said

“(c) the struggle in these models to avoid perpetuating bias and toxic speech.”

Three ideas from linguistics that everyone in AI should know, paragraph 11. The full statement is at the source.

Our reading, the sentence that was checked:

Neglecting reference, cognitive models and compositionality makes it hard for LLMs to avoid repeating bias and toxic speech.

Assessment: Unresolved Evidence as of

Where and when Gary Marcus wrote it

Marcus on AI, Three ideas from linguistics that everyone in AI should know, paragraph 11.

Said
Source observed
live page, full text, public
Evidence as of
the cutoff for this assessment
Assessment recorded
assessment marcus-three-ideas-from-linguistics-that-e640a1-a1

We recorded 26 claims from this post. See every claim from this post.

The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-06-22. This page does not claim the wording was captured then.

What the claim commits to

Kind
Causal claim
Deadline, as the speaker wrote it
None stated. This site does not add one.
Deadline state when assessed
No horizon
Deadline state on 2026-10-29
No horizon (the release’s reference date)

Conditions

  • caused by inadequate attention to reference, cognitive models, compositionality (qualifier, stated by the speaker)

From the speaker

  • inadequate attention to these three factors
  • has serious consequences
  • the struggle in these models to avoid perpetuating bias and toxic speech

Our interpretation

  • The claim attributes some difficulty avoiding bias and toxicity to inadequate reference, cognitive models and compositionality. It does not say these are the only causes.
  • Support requires evidence about that causal connection, rather than evidence that biased or toxic outputs occur.
  • The relevant systems are the language models discussed in the June 2022 post. Later research may test the proposed mechanism, but later models alone cannot establish its historical importance.

Counts as supported if: Controlled comparisons show that improving the specified capabilities materially reduces biased and toxic outputs, with competing explanations such as training data and safety fine-tuning addressed.

Counts as contradicted if: Well-controlled tests of the specified capabilities show that their inadequacy does not materially contribute to the reported difficulties. Improvement through another intervention alone is insufficient.

Retrospective: these criteria were written after the outcome was public.

Current assessment

Does the evidence support the cause?

Outcome: Unresolved

The record was checked, and the evidence available by the cutoff does not settle it.

Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.

Evidence as of
Deadline state when assessed
No horizon
Assessment recorded
Policy version
2026-10-28.7

Rationale

The research found does not settle the causal attribution. InstructGPT reduced toxicity under some conditions without a corresponding improvement in bias, but its intervention did not isolate reference, cognitive models or compositionality. That result neither proves nor disproves Marcus's proposed contribution.

Limitations

  • The search is not proof that no relevant experiment exists.
  • Successful alternative remedies would not by themselves disprove a contributing cause.

Evidence (1 item)

  • Context only: arXiv; OpenAI researchers,

    InstructGPT shows small improvements in toxicity over GPT-3, but not bias.

    Why it matters here: Human-feedback training affected the two outcomes differently. This shows why neither their occurrence nor their improvement establishes Marcus's proposed causal explanation.

    Publisher
    arXiv; OpenAI researchers
    Kind of source
    Preprint, not peer reviewed
    Describes events of
    Public from

What would settle it

Controlled studies that separately or jointly change the specified capabilities, verify those changes, and measure their effects on bias and toxicity while controlling other interventions.

Where it was said (1)

Said once in the sources this register covers.

  • : Marcus on AI, Three ideas from linguistics that everyone in AI should know

    (c) the struggle in these models to avoid perpetuating bias and toxic speech.

    Published
    2022-06-22
    Source observed
    , live page
    Access
    full text, public

History

Assessment versions

Recorded events

  1. Assessment publishedBy this site

    First assessment published after both reviews and the final-text check passed.

    About assessment marcus-three-ideas-from-linguistics-that-e640a1-a1.