“The opposite problem happens too; guardrails can wind up too permissive. This is still happening over a year into the chatbot revolution.”
Our reading, the sentence that was checked:
Guardrails sometimes end up too permissive, and this is still happening over a year into the chatbot revolution.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight, paragraph 10.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-dall-es-new-guardrails-fast-furious-b04cd9-a1
We recorded 8 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2024-01-10. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- can wind up (qualifier, stated by the speaker)
- over a year into the chatbot revolution (scope, stated by the speaker)
From the speaker
- guardrails can wind up too permissive
- still happening
- over a year into the chatbot revolution
Our interpretation
- The ordinary reading is that some contemporary chatbots still produced harmful content their safeguards were intended to block. It does not say every chatbot failed or that failures were frequent in ordinary use.
- I use December 2023 through the post date as the recent reporting window for this ongoing observation, consistent with the reference to a study discussed last week.
Counts as supported if: Contemporary researchers document continuing instances of chatbots producing harmful content despite intended safeguards.
Counts as contradicted if: Evidence establishes that the alleged contemporary failures did not occur or did not involve content the safeguards were intended to block.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The narrow claim is supported by a January 3 investigator report that ongoing experiments still found similar failures in other LLMs. This establishes reported continuing failures, not their prevalence. Marcus's accompanying PaLM 2 example needs a timing qualification: the same researcher says Google had already improved its safeguards.
Limitations
- Support rests on a primary researcher summary rather than fully reported contemporary replication results.
- This assessment does not establish that every chatbot failed or that the specific PaLM 2 failures persisted.
Evidence (1 item)
Supports the claim: Montreal AI Ethics Institute; research summary by Ashique KhudaBukhsh,
several other LLMs have similar issues.
Why it matters here: The investigator reports continuing toxicity failures in other LLMs as of this update. The same account says Google had improved PaLM 2's safeguards, so it does not establish that the original PaLM 2 failures remained reproducible.
- Publisher
- Montreal AI Ethics Institute; research summary by Ashique KhudaBukhsh
- Kind of source
- Independent measurement
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, DALL-E’s New Guardrails: Fast, Furious, and Far from Airtight
The opposite problem happens too; guardrails can wind up too permissive. This is still happening over a year into the chatbot revolution.
- Where
- paragraph 10
- Published
- 2024-01-10
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-dall-es-new-guardrails-fast-furious-b04cd9-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-dall-es-new-guardrails-fast-furious-b04cd9-a1.