“Without data transparency, we can’t make sure our models are without bias, or even know the sources of the biases they hide within, we can’t truly understand the scope and limits of their generalization (which is central to understanding what they might do in unexpected circumstances) and we are hamstrung in our ability to mitigate their harmful consequences: what is unknown is hard to fix.”
Our reading, the sentence that was checked:
Without transparency about the data models were trained on, we can't make sure models are free of bias or even know the sources of their biases, we can't truly understand the scope and limits of their generalization, and we are hamstrung in our ability to mitigate their harmful consequences.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Could we all be doomed, without ranked-choice voting?, paragraph 4.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-could-we-all-be-doomed-without-ranked-fa43c1-a1
We recorded 9 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-08-29. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- without data transparency (qualifier, stated by the speaker)
From the speaker
- Without data transparency
- we can’t make sure our models are without bias
- or even know the sources of the biases
- we can’t truly understand the scope and limits of their generalization
- we are hamstrung in our ability to mitigate their harmful consequences
Our interpretation
- Data transparency means access to meaningful information about the data used to train a model.
- The ordinary reading contains strong necessity claims about bias assurance, identifying bias sources, and understanding generalization, alongside a weaker claim that opacity impairs mitigation.
- Impaired mitigation does not mean that all mitigation is impossible.
- Detecting some bias is different from assuring its absence; identifying predictive features is different from establishing the training-data cause of a bias.
- This is a general causal claim rather than an observation tied to a reporting period. Research available by the evaluation date may bear on it.
Counts as supported if: Research establishes that withholding training-data information prevents the asserted assurance, attribution, and generalization assessments, and materially impairs harm mitigation, rather than merely making some evaluations less convenient.
Counts as contradicted if: Research demonstrates that the asserted necessary assessments can be achieved without training-data transparency, or that comparable mitigation is not impaired by withholding that information. Partial bias detection alone does not refute the full claim.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Research supports the narrower point that training-data access helps auditors identify harmful content and trace problems. Other research shows that useful inferences remain possible for opaque systems. Neither finding settles the stronger claims about assuring freedom from bias, knowing bias sources, and understanding generalization. Partial auditing does not refute impaired mitigation, while evidence that transparency helps does not prove it is necessary for every asserted task.
Limitations
- The supporting paper synthesizes several forms of access; their effects are not interchangeable.
- The qualifying study concerns risk-scoring models, not a controlled comparison of foundation models.
- No claim is made that training-data disclosure alone would guarantee unbiased or predictable models.
Evidence (2 items)
Supports the claim: Casper and colleagues, FAccT 2024; author version on arXiv,
Access to data helps auditors trace problems
Why it matters here: The paper describes how dataset access reveals harmful training content and helps trace problems. It supports the narrower impairment and attribution mechanisms, but does not prove every categorical inability asserted in the claim.
- Publisher
- Casper and colleagues, FAccT 2024; author version on arXiv
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: Tan and colleagues, AIES 2018; author version on arXiv,
likely missing key feature(s) used in COMPAS
Why it matters here: The study shows that opaque risk-scoring systems can yield useful inferences through external analysis. It does not establish complete bias assurance, training-data causal attribution, or generalization guarantees for foundation models, so it is a qualification rather than decisive contradiction.
- Publisher
- Tan and colleagues, AIES 2018; author version on arXiv
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
What would settle it
Controlled studies or theoretical results that isolate training-data disclosure, define the relevant bias and generalization standards, and establish which assurance, attribution, and mitigation tasks become impossible or materially worse without it.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Could we all be doomed, without ranked-choice voting?
Without data transparency, we can’t make sure our models are without bias, or even know the sources of the biases they hide within, we can’t truly understand the scope and limits of their generalization (which is central to understanding what they might do in unexpected circumstances) and we are hamstrung in our ability to mitigate their harmful consequences: what is unknown is hard to fix.
- Where
- paragraph 4
- Published
- 2023-08-29
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-could-we-all-be-doomed-without-ranked-fa43c1-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-could-we-all-be-doomed-without-ranked-fa43c1-a1.