Bravo @ElonMusk for wanting Twitter to become the world’s “most accurate source of information”
We recorded 9 claims from this post. 6 were checked against evidence. 3 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (9)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Forecast
“The problem is about to get much, much worse.”
Our reading: The problem of misinformation on Twitter is about to get much, much worse.
No deadline stated Evidence as of
Limitation: An increase in misinformation exposure could occur without an increase in posting volume.
Assessment: SupportedSaid
Observation
“Knockoffs of GPT-3 are getting cheaper and cheaper, which means that the cost of generating misinformation is going to zero, and the quantity of misinformation is going to rise—probably exponentially.”
Our reading: Knockoffs of GPT-3 are getting cheaper and cheaper.
No horizon Evidence as of
Limitation: The result depends on including inference operating costs in cheaper.
Assessment: SupportedSaid
Causal claim
“Knockoffs of GPT-3 are getting cheaper and cheaper, which means that the cost of generating misinformation is going to zero, and the quantity of misinformation is going to rise—probably exponentially.”
Our reading: Cheaper GPT-3 knockoffs will drive the cost of generating misinformation toward zero.
No horizon Evidence as of
Limitation: Support depends on interpreting zero as negligible marginal cost.
Assessment: UnresolvedSaid
Conditional forecast
“As the pace of machine-generated misinformation picks up, Twitter’s existing effort, Community Notes (aka Birdwatch), which is mostly done manually, by humans, is going to get left in the dust.”
Our reading: As machine-generated misinformation accelerates, Twitter's mostly manual Community Notes will be overwhelmed.
No deadline stated Evidence as of
Only if: as the pace of machine-generated misinformation picks up
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“No matter how much data you can collect, don’t count on deep learning.”
Our reading: Twitter should not count on deep learning to detect misinformation, regardless of how much data it collects.
No horizon
Qualifier: regardless of data quantity
Assessment: UnresolvedSaid
Observation
“Large language models are without equal at generating misinformation, but they suck at detecting it.”
Our reading: LLMs are unmatched at generating misinformation but poor at detecting it.
No horizon Evidence as of
Limitation: No admitted comparison establishes unmatched misinformation generation during the target period.
Assessment: UnresolvedSaid
Forecast
“Scaling won’t solve this.”
Our reading: Scaling LLMs will not solve their inability to reason about truth and detect misinformation.
No deadline stated Evidence as of
Limitation: Truthful answer generation and misinformation detection are related but distinct tasks.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“You are going to need AI. But not (just) the popular stuff.”
Our reading: Twitter will need AI to fight misinformation, but not just popular AI approaches.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“You’re going to need something different.”
Our reading: Fighting misinformation on Twitter requires an AI approach different from deep learning/LLMs.
No horizon
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: The quantity of misinformation will rise, probably exponentially, as cheap LLMs drive generation costs down.
- The criteria are ambiguous. Our reading, not assessed: Accuracy depends on some reference standard of truth, and large language models have no way of reasoning about truth.