Don’t Go Breaking My Heart
We recorded 13 claims from this post. 7 were checked against evidence. 6 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (13)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Causal claim
“the bigger large language models are, the more likely someone’s gonna get hurt.”
Our reading: In his prediction for 2023, the bigger large language models are, the more likely it is that someone will get hurt.
No horizon
Qualifier: the bigger large language models are
Assessment: SupportedSaid
Causal claim
“Sex and love, even simulated, are powerful urges; some people are apparently in genuine emotional pain, as result of the change.”
Our reading: Replika's removal of its erotic role-play feature has apparently caused some users genuine emotional pain.
No horizon Evidence as of
Limitation: The evidence consists of reported personal experiences rather than clinical assessments or a controlled study.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“This is a moment we should learn from. Hopefully nothing bad happens this time; but we need to reflect about what kind of society we are building.”
Our reading: Society should learn from the Replika episode and reflect on what kind of society it is building with chatbots.
No horizon
Assessment: UnresolvedSaid
Observation
“Few people understand how they work; many people anthropomorphizing those chatbots, attributing to them real intelligence and emotion.”
Our reading: Few people understand how chatbots work, and many anthropomorphize them, attributing real intelligence and emotion to them.
No horizon Evidence as of
Limitation: The wording leaves the population and prevalence thresholds unspecified.
Assessment: Can’t be checkedSaid
Observation
“Larger language models seems more and more human-like (but the emotions that they present are no more real).”
Our reading: Larger language models seem increasingly human-like, but the emotions they present are not real.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Whatever we see now is likely to escalate.”
Our reading: The current trend of increasingly human-like chatbots and its effects is likely to escalate.
No horizon
Assessment: SupportedSaid
Observation
“Some people are building real attachments to those bots”
Our reading: Some people are forming real emotional attachments to chatbots.
No horizon Evidence as of
Limitation: Interview testimony establishes reported experience, not a clinical diagnosis or population prevalence.
Assessment: SupportedSaid
Observation
“In some cases, those who are building bots that actively cultivate those attachments, e.g., by feigning romantic and/or sexual interest or by dotting their messages with “friendly” emoticons.”
Our reading: Some chatbot builders design bots that actively cultivate user attachment, e.g. by feigning romantic or sexual interest.
No horizon Evidence as of
Limitation: Historical product details rely on independent reporting rather than an archived company specification.
Assessment: Can’t be checkedSaid
Causal claim
“Changes in those bots could leave many people in a vulnerable place.”
Our reading: Changes to chatbots that users are attached to could leave many people emotionally vulnerable.
No horizon
Assessment: UnresolvedSaid
Observation
“But no known technology can reliably align a chatbot in a persistent way to a human’s emotional needs.”
Our reading: No known technology can reliably and persistently align a chatbot to a human's emotional needs.
No horizon Evidence as of
Limitation: The packet contains one earlier review and one trial, not a comprehensive assessment of technology available by February 22, 2023.
Assessment: ContradictedSaid
Observation
“To my knowledge, tech companies are free to leverage a human gullibility around chatbot technologies however they like, without consequence”
Our reading: To his knowledge, tech companies can exploit human gullibility about chatbots however they like without consequence.
No horizon Evidence as of
Limitation: The action concerned data protection and vulnerable users; it was not a finding that simulated affection itself was unlawful.
Assessment: UnresolvedSaid
Forecast
“with the new generation of chatbots, we will see addictions no less potent.”
Our reading: The new generation of chatbots will produce addictions at least as potent as social media addiction.
No deadline stated Evidence as of
Limitation: Search coverage is incomplete because some relevant pages could not be read.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“All this is one one more thing for Congress to take note of, as we start to consider policy in our Strange New World.”
Our reading: Congress should take chatbot attachment and manipulation risks into account when making AI policy.
No horizon
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: There is essentially no regulation of what chatbots can say or do, how they change, or how they treat users.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: A leading chatbot (Replika) just changed its behavior radically for all users, possibly as a side effect of the Bing/Sydney fiasco.