Covert racism in LLMs
We recorded 8 claims from this post. 1 was checked against evidence. 7 can’t be checked. 4 more are held. Read the post at its source.
Claims from this post (8)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“On overt measures, the systems were fine. On covert measures, they were a disaster:”
Our reading: LLMs tested looked fine on overt racism measures but did very badly on covert racism measures when prompted with African American English.
No horizon Evidence as of
Limitation: The abstract does not establish that every tested model was free of overt bias.
Assessment: Can’t be checkedSaid
Forecast
“The potential consequences on the world, given the widespread adoption of LLMs, are massive:”
Our reading: Because LLMs are so widely adopted, the potential consequences of their covert racism for the world are massive.
No horizon
Qualifier: given the widespread adoption of LLMs
Assessment: Can’t be checkedSaid
Causal claim
“LLMs are, as I have been trying to tell you, too stupid to understand concepts like people and race; their fealty to superficial statistics drives this horrific stereotyping .”
Our reading: LLMs cannot understand concepts like people and race, and their reliance on superficial statistics causes racist stereotyping.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Counterparts in other nations should consider doing the same.”
Our reading: Regulators outside the US should consider investigating LLM makers and demanding user logs.
No horizon
Assessment: Can’t be checkedSaid
Observation
“What Hofman and his collaborators (including the MacArthur Fellow Dan Jurafsky) have documented may already be having real-world impact.”
Our reading: The covert racism documented in LLMs may already be having real-world impact.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I would encourage Congress, the EEOC, HUD, the FTC, and others to investigate, and demand user logs and interaction data from the major LLM manufacturers.”
Our reading: Congress, the EEOC, HUD, the FTC and others should investigate covert racism in LLMs and demand user logs and interaction data from the major LLM makers.
No horizon
Assessment: Can’t be checkedSaid
Observation
“And Google’s debacle shows that guardrails are never easy..”
Our reading: Google's recent debacle shows that building guardrails for LLMs is never easy.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But the LLM companies should recall their systems until they can find an adequate solution.”
Our reading: LLM companies should recall their systems until they find an adequate solution to covert racism.
No horizon
Qualifier: until an adequate solution to covert racism is found
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: Scaling up LLMs makes their covert racism worse.
- The criteria are ambiguous. Our reading, not assessed: Marcus says he does not see an easy fix for covert racism in LLMs, and notes that the authors showed simply adding human feedback made it worse.
- The criteria are ambiguous. Our reading, not assessed: In many ways it is unknown how LLMs are actually used in real-world decisions such as housing, loans, and criminal proceedings, but there is now strong evidence to suspect covert racism to the extent they are used in such cases.
- The record does not meet the current rules. Our reading, not assessed: Trivial dialect choices in prompt wording appear to drive much of the covert racism in LLMs.