Things are about to get a lot worse for Generative AI
We recorded 14 claims from this post. 8 were checked against evidence. 6 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (14)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“We will publish a full report next week, but it is already clear that what we are finding poses serious challenges for generative AI.”
What we tested: Experimental findings on copyright reproduction pose serious challenges for generative AI.
Assessment: UnresolvedSaid
“OpenAI’s image software (which we accessed through Bing) is perfectly capable of verbatim and near-verbatim repetition of sources as well.”
What we tested: OpenAI's image generator can reproduce source images verbatim or near-verbatim.
Assessment: UnresolvedSaid
“And worse, infringement can happen even the user isn’t looking to infringe and doesn’t mention any character or film by name:”
What we tested: DALL-E can produce infringing images even when the user does not intend to infringe or name any character or film.
Assessment: SupportedSaid
“Generative AI systems like DALL-E and ChatGPT have been trained on copyrighted materials;”
What we tested: Generative AI systems such as DALL-E and ChatGPT were trained on copyrighted materials.
What happened: The announcement confirms that licensed contributor imagery had already been used to train DALL-E by this date. Licensed copyrighted material satisfies the claim regardless of whether its use was authorized. (Shutterstock, )
Assessment: SupportedSaid
“OpenAI, despite its name, has not been transparent about what it has been trained on.”
What we tested: OpenAI has not been transparent about its training data.
What happened: The system card describes broad source categories and filtering methods but does not identify the underlying training corpus or its substantive source inventory. This supports limited transparency, not an assertion that OpenAI disclosed nothing. (OpenAI; reproduced at dalle-3.pages.dev, )
Assessment: UnresolvedSaid
“Generative AI systems are fully capable of producing materials that infringe on copyright.”
What we tested: Generative AI systems can produce copyright-infringing materials.
Assessment: UnresolvedSaid
“They do not inform users when they do so.”
What we tested: Generative AI systems do not inform users when they produce infringing material.
Assessment: Can’t be checkedSaid
“Users may not know when they produce any given image whether they are infringing.”
What we tested: Users of generative AI may not know whether a given image they produce is infringing.
Assessment: Can’t be checkedSaid
“My guess is that none of this can easily be fixed.”
What we tested: The copyright, provenance, and transparency problems of generative AI probably cannot be easily fixed.
Assessment: UnresolvedSaid
“Unless and until somebody can invent a new architecture that can reliably track provenance of generative text and/or generative images, infringement – often not at the request of the user — will continue.”
What we tested: Unless a new architecture that reliably tracks provenance is invented, generative AI infringement, often unrequested by users, will continue.
Assessment: Can’t be checkedSaid
“A good system should give the user a manifest of sources; current systems don’t.”
What we tested: Generative AI systems should give users a manifest of sources.
Assessment: SupportedSaid
“A good system should give the user a manifest of sources; current systems don’t.”
What we tested: Current generative AI systems do not give users a manifest of sources.
What happened: The authors describe the state at their December 2023 writing date and identify the missing provenance relationships needed for attribution. This supports the contextual reading about underlying source manifests, although it is a research synthesis rather than an exhaustive product audit. (An MIT Exploration of Generative AI; author-hosted copy, )
Assessment: Can’t be checkedSaid
“In all likelihood, the New York Times lawsuit is just the first of many.”
What we tested: The New York Times lawsuit against OpenAI is likely the first of many copyright lawsuits.
Assessment: Can’t be checkedSaid
“And OpenAI faces further risks.”
What we tested: OpenAI faces further risks.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.