Is ChatGPT Really a “Code Red” for Google Search?
We recorded 13 claims from this post. 6 were checked against evidence. 7 can’t be checked. 6 more are held. Read the post at its source.
Claims from this post (13)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Causal claim
“The trouble is, those guardrails don’t come for free; the system is still shallow; ChatGPT doesn’t really know what it is it is guarding against; it mostly seems instead to be just looking for keywords, which is why it is prone to generating nonsense like this:”
Our reading: ChatGPT's guardrails don't really understand what they guard against and appear to rely mostly on keywords, which makes ChatGPT prone to producing nonsense.
No horizon
Assessment: Can’t be checkedSaid
Observation
“I like to call this sort of nonsense discomprehension: mindless answers that show that the systems has no clue what it is that is talking about.”
Our reading: ChatGPT's mindless answers ('discomprehension') show it has no clue what it is talking about.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“There’s another problem, too, arguably even bigger concern in the context of search engines: hallucination.”
Our reading: For search engines, hallucination is a further problem for LLM-based systems and arguably an even bigger concern than the shallowness, or discomprehension, he described just before.
No horizon
Scope: in the context of search engines
Assessment: UnresolvedSaid
Observation
“Once again, the result was fluent prose, and once again, hallucination ruled:”
Our reading: YouChat, though tailored for search, produced fluent but hallucinated answers to his friend's queries.
No horizon Evidence as of
Limitation: Unresolved does not mean the reported exchanges were false.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It is insidious the way that truth and falsity are so thoroughly and authoritatively mixed together. I for one am not ready for our post-truth information overlords.”
Our reading: The way truth and falsity are thoroughly and authoritatively mixed together in LLM-driven search answers like the ones he examined (such as the Neeva bio) is insidious, and he personally is not ready for 'post-truth information overlords'.
No horizon
Scope: LLM-driven search outputs like the examples he examined, such as the Neeva bio
Assessment: SupportedSaid
Observation
“Large language models are not databases. They are glommers-together-of-bits-that don’t always belong together.”
Our reading: LLMs are not databases; they combine bits of text that don't always belong together.
No horizon Evidence as of
Limitation: The cited passages do not establish a general error rate across models or tasks.
Assessment: UnresolvedSaid
Causal claim
“But as they do so, as they glom stuff together, something often gets lost in translation: which bits of text do and do not truly belong together.”
Our reading: When LLMs paraphrase and combine text, they often lose track of which pieces truly belong together, producing errors.
No horizon Evidence as of
Limitation: Summarization and truthfulness benchmarks cover different tasks and model populations.
Assessment: UnresolvedSaid
Observation
“As it happens, large language models are also wicked hard to update, typically requiring full retraining, sometimes taking weeks or months.”
Our reading: LLMs are very hard to update, usually needing full retraining that can take weeks or months.
No horizon Evidence as of
Limitation: Most relevant records describe research methods rather than representative deployment practice.
Assessment: UnresolvedSaid
Observation
“Which means for example that ChatGPT, released in November, is so old it doesn’t know who owns Twitter:”
Our reading: ChatGPT's knowledge is out of date; it doesn't know who owns Twitter.
No horizon Evidence as of
Limitation: No independent December ownership transcript was verified.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But until those bits all fit together in a way we can trust, I prefer to borrow a memorable phrase from Ariana Grande: Thank U, Next.”
Our reading: Until the pieces of LLM-based search fit together in a way that can be trusted, he personally prefers to pass on it ('Thank U, Next').
No horizon
Qualifier: until those bits all fit together in a way we can trust
Assessment: Can’t be checkedSaid
Forecast
“In the quest for a newfangled search I suspect we are once again in for a rough ride.”
Our reading: Developing LLM-based search will probably be a difficult, drawn-out process, as with driverless cars.
No horizon
Assessment: SupportedSaid
Causal claim
“Driverless cars have taken many years longer than we were originally told; outliers (AKA edge cases) have, at least so far, prevented them from making the transition from demo to widely-available reality.”
Our reading: Driverless cars have taken years longer than promised, because edge cases have so far kept them from becoming widely available.
No horizon Evidence as of
Limitation: The result depends on reading widely available as broad access beyond restricted local services.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“When it comes to LLM-driven search, caveat emptor!”
Our reading: Users should be cautious when relying on LLM-driven search.
No horizon
Held from this post (6)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- Attribution is uncertain
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: LLMs are text predictors, essentially turbocharged autocomplete, not databases.
- The record does not meet the current rules. Our reading, not assessed: Hybrid search/LLM systems are only at proof-of-concept stage and are nothing like systems that can be relied upon.
- The record does not meet the current rules. Our reading, not assessed: Perplexity.ai and you.com are exploring an interesting idea in hybrids of classical search and LLMs, which may allow swifter updates.