They're Looting The Internet
We recorded 17 claims from this post. 5 were checked against evidence. 12 can’t be checked. 12 more are held. Read the post at its source.
Claims from this post (17)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: ContradictedSaid
Observation
“As a result, every single LLM-based application — ChatGPT, Google Gemini, Anthropic's Claude, Meta's LLaMA, to name but a few — is trained on an indeterminately-large portion of the internet made up of both Common Crawl (a 250-billion page repository of web content) and what appears to be as much of the rest of the internet as they're able to download, legally or otherwise.”
Our reading: Every single LLM-based application, including ChatGPT, Google Gemini, Anthropic's Claude and Meta's LLaMA, is trained on an indeterminately large portion of the internet made up of Common Crawl and what appears to be as much of the rest of the internet as they are able to download, legally or otherwise.
No horizon Evidence as of
Limitation: The result addresses the explicit universal scope, not whether web-derived training is common.
Assessment: ContradictedSaid
Observation
“These models are not "learning" or, let's be honest, "training" on this data, because that's not how they work — they're using mathematics to plagiarize it based on the likelihood that somebody else's answer is the correct one.”
Our reading: LLMs do not learn from data; they plagiarize it mathematically, going by the likelihood that someone else's answer is correct.
No horizon Evidence as of
Limitation: The result uses the ordinary machine-learning meaning of learning.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“While OpenAI, Google and Meta would like to claim that these are "publicly-available" works that they are "training on," the actual word for what they're doing is "stealing."”
Our reading: Although OpenAI, Google and Meta call it training on 'publicly-available' works, the right word for what they are doing is 'stealing'.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Every single tech company making a LLM is stealing, justifying it by using the previous model of the internet where everything published online was there for the taking and conflating access to content with ownership in the process.”
Our reading: Every single tech company making an LLM is stealing, justifying it with the earlier model of the internet in which everything published online was there for the taking, and conflating access to content with ownership.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“LLMs are a globally-perpetuated act of theft taking place in broad daylight, to the point that OpenAI has told England's House of Lords that "it would be impossible to train today's leading AI models without using copyrighted materials."”
Our reading: LLMs are a globally perpetuated act of theft taking place in broad daylight.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“The generative AI boom has exactly the same stench as the metaverse, and it's happening for exactly the same reason — the people making the products are not building things for human beings, but to show the markets that they'd continue to grow.”
Our reading: Like the metaverse, the generative AI boom is driven by the need to show markets continued growth rather than to build things for people.
No horizon
Assessment: Can’t be checkedSaid
Observation
“OpenAI is no different. After all, it’s effectively a subsidiary of Microsoft.”
Our reading: OpenAI is effectively a subsidiary of Microsoft and, like the others, is a thief rather than an innovator.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“And when you have an industry piloted by people who don't make products for people, you don't create anything useful, and you're doomed to make the same stupid, obvious mistakes.”
Our reading: When an industry is piloted by people who don't make products for people, it doesn't create anything useful and is doomed to make the same stupid, obvious mistakes.
No horizon
Only if: an industry piloted by people who don't make products for people
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Even the claims of what generative AI has done need to be taken with a grain — or a spoonful — of salt.”
Our reading: Claims about what generative AI has achieved should be treated with heavy skepticism.
No horizon
Assessment: Can’t be checkedSaid
Rhetoric
“I’d say it has the engineering ability of a first-year CompSci undergraduate, but that would be unfair to first-year CompSci undergraduates.”
Our reading: Devin's engineering ability is below that of a first-year computer science undergraduate.
No horizon
Assessment: SupportedSaid
Conditional forecast
“As I wrote (and broadcast) recently, LLMs are running out of training data, and the desperation to find more will lead to significant legal consequences when it's revealed exactly how much they stole and from whom they stole it from.”
Our reading: The hunt for more training data will bring AI companies significant legal consequences once it is revealed how much they stole and from whom.
Settled before its deadline Evidence as of
Only if: when it's revealed how much they stole and from whom
Assessment: UnresolvedSaid
Forecast
“Yet I believe generative AI is going to lead several of these companies to ruin.”
Our reading: Generative AI will lead several big tech companies to ruin.
No deadline stated Evidence as of
Limitation: The search does not constitute a comprehensive insolvency audit of every company mentioned in the post.
Assessment: UnresolvedSaid
Observation
“LLMs also tend to hallucinate, a virtually-unsolvable problem where they authoritatively make incorrect statements that creates horrifying results in generative art and renders them too unreliable for any kind of mission critical work.”
Our reading: LLM hallucination is a virtually unsolvable problem that makes LLMs too unreliable for any mission-critical work.
No horizon Evidence as of
Limitation: This evaluates the state described in early 2024, not the capabilities of later models.
Assessment: Can’t be checkedSaid
Observation
“Generative AI's energy demands are unsustainable, as are its compute demands”
Our reading: Generative AI's energy and compute demands are unsustainable.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“The answer is obvious: because they believe it'll make them money.”
Our reading: The tech industry is pushing generative AI on people because it believes generative AI will make it money.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Yet I believe that they're all disconnected from any actual value creation.”
Our reading: He believes Google, Meta, Microsoft and OpenAI are all disconnected from any actual value creation.
No horizon
Qualifier: I believe
Assessment: Can’t be checkedSaid
Forecast
“Nothing — and I repeat, nothing — good will come from this.”
Our reading: Nothing good will come from Google's leadership shuffle that moved senior AI engineer Cheenu Venkatachary to lead the ranking and search quality teams.
No horizon
Held from this post (12)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: Google has not found a way to make money from its generative search product, which still hallucinates, and is thinking about charging for it.
- The two reviews reached different assessments. Our reading, not assessed: In its demonstrated Upwork task, Devin produced code that missed requirements, used outdated approaches, and took longer than a human would.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: By all accounts, generative AI companies are already ingesting AI-generated content, as when Adobe's Firefly AI accidentally ingested generative art that had found its way into the Adobe Stock image pool.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: As generative AI companies grow desperate for data, they will ingest AI-generated content, causing model collapse that destroys their models' usefulness.
- The two reviews read the claim differently