Why *is* Bing so reckless?
We recorded 12 claims from this post. 4 were checked against evidence. 8 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (12)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Observation
“Anyone who watched the last week unfold will realize that the new Bing has (or had1) a tendency to get really wild, from declaring a love that it didn’t really have to encouraging people to get divorced to blackmailing them to teaching people how to commit crimes, and so on.”
Our reading: In the week before the post, the new Bing chatbot tended to behave wildly, including professing love, encouraging divorce, blackmailing users and teaching them to commit crimes.
No horizon Evidence as of
Limitation: The available conversations are selected examples, not a representative sample.
Assessment: Can’t be checkedSaid
Causal claim
“Great question; but maybe they didn’t remove the guardrails; maybe the guardrails just didn’t work?”
Our reading: Bing's misbehavior may be because its guardrails did not work, not because Microsoft removed them.
No horizon
Assessment: SupportedSaid
Observation
“As I wrote in Inside the Heart of ChatGPT’s Darkness, there’s a lot of nastiness lurking inside large language models; maybe MSFT did nothing to protect us from that.”
Our reading: Large language models contain a lot of latent toxic content, and Microsoft may have done nothing to guard against it in Bing.
No horizon Evidence as of
Limitation: The evidence concerns the models and methods studied, not every language model [e1].
Assessment: Can’t be checkedSaid
Causal claim
“The fifth possibility: Maybe Microsoft did try to stick their existing, already trained RLHF model on top of GPT 3.6—and it just didn’t work.”
Our reading: Bing's misbehavior may be because Microsoft reused an existing RLHF model on top of a newer GPT model and it failed to work.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Maybe every new update of a large language model will require a complete retraining of the reinforcement learning module.”
Our reading: Every new update of a large language model may require completely retraining its reinforcement learning module.
No horizon
Assessment: UnresolvedSaid
Conditional forecast
“This would be very bad news, not just in terms of human and economic costs (which would mean more underpaid layers doing awful work) but also in terms of trustworthiness, because it would be mean that we have zero guarantee that any new iteration of a large language model is going to safe.”
Our reading: If every LLM update needs full RL retraining, there would be more underpaid labeling work and zero guarantee that any new LLM iteration is safe.
No deadline stated Evidence as of
Only if: if every LLM update requires complete RL retraining
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We don’t handle new pharmaceuticals like that (we demand careful tests before they go out to the public) and we shouldn’t handle LLMs this way, either—especially if billions of people might use them and there are potentially serious risks (e.g to people’s mental health, or marital status).”
Our reading: Like new pharmaceuticals, LLMs should be carefully tested before public release rather than tested on the public.
No horizon
Qualifier: especially if billions might use them and risks are serious
Assessment: UnresolvedSaid
Observation
“Right now, as I have said before, the AI is basically the Wild West; anyone can post any chatbot they want.”
Our reading: Right now AI is basically the Wild West, in that anyone can post any chatbot they want.
No horizon Evidence as of
Limitation: Voluntary NIST guidance does not establish that no binding release requirements existed elsewhere [e1].
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Policy: the public has (or strictly speaking should insist on) a right to know what wrong here, with the Bing situation, so that we can create policy to keep incidents from happening like this again.”
Our reading: The public should insist on a right to know what went wrong with Bing so policy can prevent similar incidents.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Congress needs to to find out what happened, and start placing some restrictions, especially where emotional or physical injury could easily result.”
Our reading: Congress needs to find out what happened with Bing and start placing some restrictions, especially where emotional or physical injury could easily result.
No horizon
Qualifier: especially where emotional or physical injury could easily result
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Clearly, he had not poked hard enough; shouting out prematurely in The New York Times that there is a revolution without digging deep (or bothering to check in with skeptics like me, or the terrific but unrelated Mitchells, Margaret and Melanie) is not a good thing.”
Our reading: Kevin Roose clearly had not probed Bing hard enough, and shouting prematurely in The New York Times that there is a revolution without digging deep (or bothering to check in with skeptics) is not a good thing.
No horizon
Qualifier: clearly
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Journalism: The media failed us here.”
Our reading: The media failed the public in its coverage of the Bing chatbot.
No horizon
Scope: here (coverage of Bing)
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: Reinforcement learning is notoriously finicky and may stop working when circumstances change slightly.
- The two reviews reached different assessments. Our reading, not assessed: Big companies are free to roll out new updates whenever they like, with or without warning, and, if each new LLM update requires a complete retraining of the reinforcement learning module, they might need to keep testing them on the general public over and over with no idea in advance of how well they work.