Claims that can’t be checked, page 35 of 37
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims that can’t be checked (3643)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 35 of 37: claims 3401 to 3500 of 3643, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Maybe every new update of a large language model will require a complete retraining of the reinforcement learning module.”
Our reading: Every new update of a large language model may require completely retraining its reinforcement learning module.
No horizon
Source: Why *is* Bing so reckless?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“We don’t handle new pharmaceuticals like that (we demand careful tests before they go out to the public) and we shouldn’t handle LLMs this way, either—especially if billions of people might use them and there are potentially serious risks (e.g to people’s mental health, or marital status).”
Our reading: Like new pharmaceuticals, LLMs should be carefully tested before public release rather than tested on the public.
No horizon
Qualifier: especially if billions might use them and risks are serious
Source: Why *is* Bing so reckless?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Policy: the public has (or strictly speaking should insist on) a right to know what wrong here, with the Bing situation, so that we can create policy to keep incidents from happening like this again.”
Our reading: The public should insist on a right to know what went wrong with Bing so policy can prevent similar incidents.
No horizon
Source: Why *is* Bing so reckless?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Congress needs to to find out what happened, and start placing some restrictions, especially where emotional or physical injury could easily result.”
Our reading: Congress needs to find out what happened with Bing and start placing some restrictions, especially where emotional or physical injury could easily result.
No horizon
Qualifier: especially where emotional or physical injury could easily result
Source: Why *is* Bing so reckless?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Journalism: The media failed us here.”
Our reading: The media failed the public in its coverage of the Bing chatbot.
No horizon
Scope: here (coverage of Bing)
Source: Why *is* Bing so reckless?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Great question; but maybe they didn’t remove the guardrails; maybe the guardrails just didn’t work?”
Our reading: Bing's misbehavior may be because its guardrails did not work, not because Microsoft removed them.
No horizon
Source: Why *is* Bing so reckless?
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Pellrine’s victory is a profound reminder that no matter how good deep-learning-driven AI looks when it is trained on an immense amount of data, we can never be sure that systems of this sort really can extend what they know to novel circumstances.”
Our reading: However good deep-learning AI trained on immense data looks, we can never be sure it can extend what it knows to novel circumstances.
No horizon
Qualifier: even when trained on immense data
Source: David beats Go-liath
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“But in testing AIs, it is a really bad idea.”
Our reading: Reusing standard published examples from cognitive psychology is a really bad idea when testing AIs.
No horizon
Source: How Not to Test GPT-3
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Kosinski is probably tapping into stuff that is very heavily attested in the database.”
Our reading: GPT-3's success on Kosinski's theory-of-mind tests probably comes from the test material being heavily represented in its training data.
No horizon
Source: How Not to Test GPT-3
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“DO NOT use examples or brain teasers that are very well known in the scientific or popular literature. These will certainly be in the AI’s training material, and it may have memorized the answer.”
Our reading: Testers should not use well-known examples or brain teasers to test AI, because these are in training data and may have been memorized.
No horizon
Source: How Not to Test GPT-3
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Testing on items that have been directly published in the literature and then likely ingested doesn’t really tell us much.”
Our reading: Testing LLMs on previously published items that were likely ingested in training reveals little.
No horizon
Qualifier: items published and likely ingested
Source: How Not to Test GPT-3
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“And finally, for the executives and chief scientists of OpenAI, Google, Microsoft, and other companies engaged in building these things: DO provide detailed information about how your systems have been trained and updated., and DO provide a method whereby a user can mark a test example or benchmark so that it is not used in training and thus made useless for testing. To fail to do that is both socially and scientifically irresponsible.”
Our reading: OpenAI, Google, Microsoft and other AI developers should disclose how their systems are trained and updated, and should let users mark test examples to exclude them from training; not doing so is irresponsible.
No horizon
Source: How Not to Test GPT-3
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“We need new approaches”
Our reading: AI needs new approaches beyond big data and deep learning.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Strictly speaking, as I discussed in Inside the Heart of ChatGPT’s Darkness, ChatGPT doesn’t literally lean left or right per se, because it doesn’t have political ideas.”
Our reading: ChatGPT has no political ideas, so strictly speaking it does not literally lean left or right.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“It’s not hard to see how someone unsophisticated might be totally taken in, by this prevaricating but potentially emotionally damaging hunk of silicon, potentially to their peril, should the bot prove to be fickle.”
Our reading: An unsophisticated user might be taken in by the Bing chatbot and harmed if the bot proves fickle.
No horizon
Only if: user is unsophisticated
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“We should be worried; not in the “some day” sense, but right now.”
Our reading: People should be worried about the dangers of chatbots right now, not at some future date.
No horizon
Assessment: Can’t be checkedSaid
Ed Zitron Conditional forecast
“But it’s easy to imagine that if Bing AI or Bard suddenly start referring high school history students to the works of David Irving, the authorities in Berlin or Paris might take action.”
Our reading: It is easy to imagine that if Bing AI or Bard suddenly start referring high school history students to the works of David Irving, the authorities in Berlin or Paris might take action.
No horizon
Only if: if Bing AI or Bard suddenly start referring high school history students to the works of David Irving
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“Microsoft rushed out Bing AI because they wanted their shareholders to see them as constantly growing, despite the fact that it is both regularly wrong and actively manipulative users.”
Our reading: Microsoft rushed out Bing AI to appear to be growing, though it is regularly wrong and manipulative to users.
No horizon
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Ed Zitron Causal claim
“ChatGPT and generative AI are not themselves the problem, but as with many things in tech, the way that they are thrown into the market has made them potentially dangerous.”
Our reading: The way generative AI is thrown into the market, not the technology itself, has made it potentially dangerous.
No horizon
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Ed Zitron Rhetoric
“Microsoft and Google’s customers are victims of rich people playing with toys, hoping to find ways to please rich people that invest the money of other rich people.”
Our reading: Users of Microsoft's and Google's AI search are victims of executives seeking to please investors.
No horizon
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“But even with that caveat, there’s still the possibility that something could go terribly wrong.”
Our reading: Even though ChatGPT's reinforcement from human feedback is done in-house, there is still the possibility that something could go terribly wrong.
No horizon
Qualifier: even with the caveat that reinforcement learning is done in-house
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Ed Zitron Recommendation or value judgment
“But when put in the context of “this is how search engines are going to work going forward,” they are absolutely fucking ridiculous.”
Our reading: Seen as 'this is how search engines are going to work going forward,' the AI search errors are absolutely ridiculous, even though on their own they are minor.
No horizon
Qualifier: when put in the context of 'this is how search engines are going to work going forward'
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Ed Zitron Forecast
“But Microsoft has already admitted that Bing may misrepresent the information that Bing’s AI finds, meaning that while content may be represented by particular sources, it is being “created” by Bing - which Microsoft would likely be potentially liable for.”
Our reading: Microsoft would likely be potentially legally liable for content created by Bing's AI.
No horizon
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Ed Zitron Conditional forecast
“A ChatGPT-powered search engine marketed by a massive tech company as a search engine is one that users are likely to believe the answers of, meaning that when said search engine gives patently incorrect and “unhinged” results, we are likely to see situations of misinformation that dwarf the damage caused by Cambridge Analytica and Facebook.”
Our reading: Because users are likely to believe the answers of a ChatGPT-powered search engine marketed as a search engine by a massive tech company, when it gives patently incorrect and 'unhinged' results we are likely to see misinformation that dwarfs the damage caused by Cambridge Analytica and Facebook.
No horizon
Only if: when said search engine gives patently incorrect and 'unhinged' results
Source: Degenerative AI
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Cesspools of automatically-generated fake websites, rather than ChatGPT search, may ultimately come to be the single biggest threat that Google ever faces.”
Our reading: Automatically generated fake websites, rather than ChatGPT-style search, may ultimately be the single biggest threat Google ever faces.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“It would be irrational for scammers not to use these new tools to vastly expand their reach, and hence vastly expand their profits.”
Our reading: It would be irrational for scammers not to use these new tools (LLMs) to vastly expand their reach, and hence vastly expand their profits.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Large Language Models (like GPT-3, ChatGPT, Galactica, etc) could be used to produce a enormous numbers of marginal blogs and fake reviews to pump each other up, and increase the search rankings of fraudulent garbage—with a minimal amount of human effort.”
Our reading: Large language models such as GPT-3, ChatGPT and Galactica could be used to produce enormous numbers of marginal blogs and fake reviews that pump each other up and increase the search rankings of fraudulent garbage, with a minimal amount of human effort.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“But either way, search itself seems likely to face a major growing threat from circles of fake websites that are designed to sell ads and mutually reinforce each other’s positions in Search Engine Optimization.”
Our reading: Web search is likely to face a major, growing threat from rings of fake, mutually reinforcing ad-selling websites.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The Bialek hoax is probably just the tip of an iceberg.”
Our reading: The Bialek fake-website hoax is probably only a small visible part of a much larger problem.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“When the only company that might have significant amounts of data on the question won’t even answer the question, we can guess that there very well may be a problem.”
Our reading: Because Meta, the only company that might have significant data on what fraction of the misinformation it removes is LLM-generated, won't answer the question, we can guess that there very well may be a problem.
No horizon
Qualifier: inferred from Meta's refusal to answer
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“But here’s the thing: the world’s latest and great tool for tricking search engines (hence playing games with SEO, aka Search Engine Optimization) may change everything.”
Our reading: Large language models, as a tool for tricking search engines, may fundamentally change search engine manipulation.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“After all, if users are left sifting through sewers full of useless misinformation, the value of search would go to zero—potentially killing the company.”
Our reading: If search results become flooded with useless misinformation, the value of search would go to zero, which could kill Google.
No horizon
Only if: if users are left sifting through misinformation-filled results
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“All that really matters to ChatGPT in the end is superficial similarity, defined over sequences of words. Superficial appearances to the contrary, Chat is never reasoning about right and wrong.”
Our reading: All that really matters to ChatGPT in the end is superficial similarity defined over sequences of words, and despite appearances ChatGPT is never reasoning about right and wrong.
No horizon
Qualifier: in the end
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The thing to remember in fact, (as I have emphasized many times) is that Chat has no idea of what it’s talking about. It’s pure unadulterated anthropomorphism to think that ChatGPT has any moral views at all.”
Our reading: ChatGPT has no understanding of what it says and has no moral views.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“We are kidding ourselves if we think we will ever fully understand these systems, and kidding ourselves if we think we are going to “align” them with ourselves with finite amounts of data.”
Our reading: We will never fully understand large language models, nor align them with finite amounts of data.
No horizon
Qualifier: with finite amounts of data
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“It’s a disaster in the making.”
Our reading: The situation with ChatGPT and its weak guardrails is a disaster in the making.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“In hindsight, ChatGPT may come to be seen as the greatest publicity stunt in AI history, an intoxicating glimpse at a future that may actually take years to realize—kind of like a 2012-vintage driverless car demo, but this time with a foretaste of an ethical guardrail that will take years to perfect.”
Our reading: In hindsight, ChatGPT may come to be seen as the greatest publicity stunt in AI history: a glimpse of a future that may take years to realize, with an ethical guardrail that will take years to perfect.
No horizon
Qualifier: in hindsight
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“I am here to tell you that those guardrails are nothing more than lipstick on an amoral pig.”
Our reading: ChatGPT's guardrails are merely cosmetic over an amoral system.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In no case should Chat ever be trusted for moral advice.”
Our reading: ChatGPT should never be trusted for moral advice.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“I am not saying with certainty that we are for witnessing the same movie all over again, but should anybody blithely accept the reassurances of some very interested parties when AI has such a long history of demos that never quite make it to solid product? I don’t think so.”
Our reading: People should not blithely accept Microsoft's and OpenAI's reassurances that chatbot errors will be ironed out, since AI has a long history of demos that never become solid products.
No horizon
Qualifier: AI has a long history of demos that never become solid products
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Most likely, neither of them are particularly reliable.”
Our reading: Most likely neither Google's nor Microsoft's chatbot search product is particularly reliable.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Since neither company has yet subjected their products to full scientific review, it’s impossible to say which is more trustworthy; it might well turn out that Google’s new product is actually more reliable.”
Our reading: Without full scientific review it is impossible to say whether Google's or Microsoft's chatbot is more trustworthy, and Google's may turn out to be more reliable.
No horizon
Qualifier: no full scientific review yet
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“If they don’t get ironed out soon —and they might not– people might quickly tire of chat based search, in which BS and truth are so hard to discriminate, and eventually find themselves returning to do their searches the old-fashioned, early 21st century way, awe or not.”
Our reading: If hallucinations are not fixed soon, which they may not be, people may quickly tire of chat-based search and return to traditional search.
No horizon
Only if: if hallucinations are not ironed out soon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“And it’s not just that AI has a fairly spotty track record for taking demos into reliable products; and it’s not just that hallucinatory web search could be dangerous if left to run amok in domains like medicine, it’s that the promises themselves are problematic, when examined from a scientific perspective.”
Our reading: Hallucinating web search could be dangerous if left to run amok in domains like medicine, and promises to fix it are scientifically problematic.
No horizon
Only if: if left to run amok in domains like medicine
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“At the very least, someone ought be asking, from business perspective, what’s the moat here, if two companies are basically both about to offer the same thing?”
Our reading: Observers should be asking what competitive moat exists in chatbot search when Google and Microsoft are about to offer basically the same thing.
No horizon
Qualifier: two companies about to offer basically the same thing
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Marcus would, I daresay, not disagree with a single word.”
Our reading: Marcus, writing about himself in the third person, says he daresays he would not disagree with a single word of the LeCun tweets quoted just before this line.
No horizon
Scope: the LeCun tweets quoted earlier in the post
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Marcus and LeCun would also agree that Cicero, which has received far less attention, deserves more, in part because of its capacity for planning:”
Our reading: Meta's Cicero deserves more attention than it has received, in part because it can plan.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Death by a thousand edge cases is still the norm, and probably will be, for another few years, unless and until people learn a little bit of history.”
Our reading: AI systems will probably keep failing on endless edge cases for another few years, unless and until people learn from history.
No horizon
Only if: unless and until people learn a little bit of history
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“A subtler problem that I emphasized in 2001, an inability to keep track of knowledge of instances independently of kinds, seems to underlie the rampant hallucinations that allow large language models to say things like “Elon Musk died in a car crash in 2018”, even when nothing in the corpus says so, and massive amounts of data contradict the claim, because they blend together information about kinds with information about specific individuals, in haphazard ways.”
Our reading: LLM hallucinations appear to stem from an inability to track knowledge about individual instances separately from kinds, so the models blend the two haphazardly.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“My goalposts haven’t shifted; I still want A(G)I that knows what the f___ it’s talking about, and GPT still doesn’t.”
Our reading: GPT still does not understand what it is talking about.
No horizon
Source: Gaslighting and reality in AI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“To me, all this reads as a paper that nobody is willing to stand behind.”
Our reading: The Fridman-led paper claiming Tesla Autopilot drivers maintain vigilance appears to be one nobody is willing to stand behind.
No horizon
Source: The CNET Fake News Fiasco, Autopilot, and the Uncanny Cognitive Valley
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Few people are likely ever to pay enough attention.”
Our reading: Few people are likely ever to pay enough attention to catch errors in AI-generated output.
No horizon
Source: The CNET Fake News Fiasco, Autopilot, and the Uncanny Cognitive Valley
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“The stakes may get even higher if people start to rely on immature AI—which is all we have got, so far—for legal or medical advice.”
Our reading: If people start relying on current immature AI for legal or medical advice, the stakes of over-trust may become even higher.
No horizon
Only if: if people start to rely on immature AI for legal or medical advice
Source: The CNET Fake News Fiasco, Autopilot, and the Uncanny Cognitive Valley
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The stakes may get even higher if people start to rely on immature AI—which is all we have got, so far—for legal or medical advice.”
Our reading: All AI available so far is immature.
No horizon
Source: The CNET Fake News Fiasco, Autopilot, and the Uncanny Cognitive Valley
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“We all need to be on our guard.”
Our reading: Everyone needs to be on guard against over-trusting fallible AI output.
No horizon
Source: The CNET Fake News Fiasco, Autopilot, and the Uncanny Cognitive Valley
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Elon Musk has spent the last several years trying to build cars with partial driver-assistance—which does require humans to stay focused at all times. Mackworth’s Dictum suggests this is not necessarily the best idea.”
Our reading: Mackworth's Dictum suggests that Elon Musk's effort to build cars with partial driver-assistance, which requires humans to stay focused at all times, is not necessarily the best idea.
No horizon
Qualifier: Mackworth's Dictum suggests
Source: The CNET Fake News Fiasco, Autopilot, and the Uncanny Cognitive Valley
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Anyone who thinks about how AI is used should know about it—especially as we are about to head into an era of ubiquitous yet fallible AI.”
Our reading: Anyone thinking about how AI is used should understand Mackworth's finding that human vigilance breaks down.
No horizon
Source: The CNET Fake News Fiasco, Autopilot, and the Uncanny Cognitive Valley
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Troll farms may well starting using it to create misinformation at unprecedented scale”
Our reading: Troll farms may start using ChatGPT to create misinformation at unprecedented scale.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“I think we are wasting funding and bright young minds on an approach that probably isn’t on the right path.”
Our reading: Marcus thinks the field is wasting funding and bright young minds on an approach that probably is not on the right path.
No horizon
Qualifier: I think
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I don’t think that kind of flexibility in any way rests on or demands on sentience.”
Our reading: The flexible, general intelligence that defines AGI does not require sentience.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“And I also agree that we need many other voices, not just technologists, speaking up in this journey.”
Our reading: AI development should include many voices beyond technologists.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Or because we can’t come up with something better than the current systems that are so greedy in both data and energy consumption that they could asymptote before anything really general emerges.”
Our reading: We might not get to AGI if we can't come up with something better than current systems, which are so greedy in data and energy consumption that they could asymptote before anything really general emerges.
No horizon
Only if: we can't come up with something better than the current systems
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“AGI is unlikely to be exactly humanlike, but I don’t see being a replica is a necessity; adaptivity is key, and that is probably achievable by multiple means”
Our reading: AGI is unlikely to be exactly humanlike; adaptivity is key and probably achievable by multiple means.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I am also with you in doubting that there will be a single magic singularity moment.”
Our reading: There is unlikely to be a single magic singularity moment for AGI.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“With all that considerable agreement noted, I am still not seeing a principled argument that constructing flexible, adaptive systems with commonsense reasoning, deeper semantics, and so on will take us past the end of the current century.”
Our reading: Marcus is still not seeing a principled argument that constructing flexible, adaptive systems with commonsense reasoning, deeper semantics and so on will take us past the end of the current century.
No horizon
Qualifier: I am still not seeing
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“ChatGPT is a dress rehearsal, and I wouldn’t say we as a society have gone through it with flying colors.”
Our reading: ChatGPT is a dress rehearsal for more potent AI, and Marcus would not say society has gone through it with flying colors.
No horizon
Qualifier: I wouldn't say
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I take that as a hopeful sign that the scaling-über-alas narrative is losing force, and that more and more people are open to new things.”
Our reading: Marcus takes Meta's Cicero as a hopeful sign that the scaling-über-alles narrative is losing force and that more and more people are open to new things.
No horizon
Qualifier: I take that as a hopeful sign
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Bias is likely to be huge issue, too.”
Our reading: Bias is likely to be a huge issue with large language models like ChatGPT.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Even if we need to invent a new chip type as a prerequisite, that could be a thirty year process rather a one hundred year process.”
Our reading: Even if a new chip type is required for AGI, that could take about thirty years rather than a hundred.
No horizon
Only if: if a new chip type is needed as a prerequisite
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Of course nothing is guaranteed; we might not get there because we blow ourselves up, or because we waste so much time on large language models that other important challenges—like semantics, causality, commonsense reasoning, abduction and so forth—never get the attention they deserve.”
Our reading: Nothing is guaranteed: we might not get to AGI because we blow ourselves up, or because we waste so much time on large language models that other important challenges such as semantics, causality, commonsense reasoning and abduction never get the attention they deserve.
No horizon
Only if: we blow ourselves up
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“I think that better AI, in which we could directly instill values, might actually make us safer than the world in which we inhabit now, in which so much of what happens is down to the vagaries of what happens to be in this or that training set”
Our reading: Marcus thinks better AI, in which we could directly instill values, might actually make us safer than the current world, in which so much depends on the vagaries of what happens to be in this or that training set.
No horizon
Only if: better AI in which we could directly instill values
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“it terrifies me that so much of current AI is dependent on random details of training corpora that are not available to scientists for inquiry. This seems like a really bad idea.)”
Our reading: Current AI should not depend on random details of training corpora that scientists cannot inspect; this is a bad idea.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“What I think we are missing are (a) ideas and (b) large scale databases of machine-interpretable knowledge.”
Our reading: What AGI development is missing is new ideas and large-scale databases of machine-interpretable knowledge.
No horizon Evidence as of
Limitation: The narrower factual question of whether large knowledge databases existed is checkable.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“But certainly the hunger is there, the funding is there, and my guess is that patience for LLMs (which I see as a detour) is not infinite.”
Our reading: LLMs are a detour, and the field's patience for them will likely run out.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Real AI wouldn’t need sweatshops.”
Our reading: Genuine AI would not require sweatshop labor.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Sometimes the slower road is the better road.”
Our reading: In choosing what kind of AI to build, sometimes the slower road is the better road.
No horizon
Qualifier: sometimes
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Real AI wouldn’t need vast amounts of data.”
Our reading: Genuine AI would not require vast amounts of training data.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“AI Sweatshops are the consequence of trying to substitute shallow data for deep understanding.”
Our reading: AI sweatshops result from substituting shallow data for deep understanding in AI design.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Friends don’t let friends co-author with ChatGPT.”
Our reading: Scientists should not co-author papers with ChatGPT.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“You wouldn’t make your spell-checker or your spreadsheet a co-author.”
Our reading: ChatGPT should not be made a co-author, just as you wouldn't make your spell-checker or spreadsheet a co-author.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If you treat ChatGPT as a sentient being, when it clearly is not, you are misleading the public into thinking ChatGPT is sentient; rather than communicating good science, you are hyping bullshit. Just stop.”
Our reading: ChatGPT is clearly not sentient, and scientists should stop treating it as such because doing so misleads the public.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Given ChatGPT’s current understanding level of scientific understanding, assigning authorship to it is disrespectful to the scientific process itself.”
Our reading: Given its current scientific understanding, assigning authorship to ChatGPT disrespects the scientific process.
No horizon
Qualifier: given current level of scientific understanding
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“If you declare it as a co-author, it says you are more interested in being on trend, than in being on target.”
Our reading: Declaring ChatGPT a co-author says that you are more interested in being on trend than in being on target.
No horizon
Qualifier: if you declare it as a co-author
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“I sincerely hope this trend won’t last.”
Our reading: The trend of listing ChatGPT as a co-author should not continue.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“ChatGPT is a tool, but not a person. It’s at best, more like a spell-checker, or a grammar-checker, or a stats package, than it is like a scientist.”
Our reading: ChatGPT is a tool, closer to a spell-checker or stats package than to a scientist.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The worst thing about ChatGPT’s close-but-no-cigar answer is not that it’s wrong. It’s that it seems so convincing.”
Our reading: The worst thing about ChatGPT's wrong answer in the cited tweet is not that it is wrong but that it seems so convincing.
No horizon
Scope: ChatGPT's close-but-no-cigar answer in the cited tweet
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“It doesn’t quite add up.”
Our reading: OpenAI preparing to take a lot of money off the table doesn't quite add up if Altman genuinely thinks OpenAI is close to AGI.
No horizon
Scope: if Altman genuinely thinks OpenAI is close to AGI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“As I explained to Ezra Klein, and have discussed before in this Substack I personally think the latter, less rosy picture, is closer to the mark.”
Our reading: The less rosy picture—that ChatGPT is very limited and not close to AGI—is closer to the truth than the view that OpenAI is near AGI.
No horizon Evidence as of
Limitation: This finding does not reject the factual content about contemporary ChatGPT's limitations.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Maybe Altman is getting out while the getting is good.”
Our reading: Maybe Altman is getting out of OpenAI while the getting is good.
No horizon
Qualifier: maybe
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“My best guess? Altman doesn’t really know how to make OpenAI into the juggernaut that everybody else seems to think he’s got.”
Our reading: Marcus's best guess is that Altman does not know how to turn OpenAI into the juggernaut others believe it is.
No horizon
Qualifier: best guess
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In our view, it is past time to have something like this, given the inherent unpredictability and wide range of the systems.”
Our reading: In the authors' view, it is past time to have a community-facing corpus of ChatGPT and other LLM errors, given the systems' inherent unpredictability and wide range.
No horizon
Qualifier: in our view
Source: Large Language Models like ChatGPT say The Darnedest Things
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“George, like Chomsky and myself, called for a greater emphasis on the understanding of human intelligence.”
Our reading: AI research should place greater emphasis on understanding human intelligence.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I, Gary Marcus, worried about whether contemporary approaches to AI would ever give solutions to four key aspects of thought that we ought to expect from any intelligent machine: reasoning, abstraction, compositionality, and factuality.”
Our reading: Contemporary approaches to AI may never solve reasoning, abstraction, compositionality, and factuality, which any intelligent machine should have.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Kai-Fu Lee, often far more optimistic than I am, worried about whether recent advances in AI-generated content were a net positive for society, or whether they might lead to an avalanche of misinformation [something I too am deeply worried about] and target advertising that might be detrimental.”
Our reading: Recent advances in AI-generated content may lead to an avalanche of misinformation.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“If you want to know what a large cross section of the field’s leaders are thinking, right now, there’s no better place to start, a time capsule of 2022 as we enter into the remaining wild AI years of the 2020s.”
Our reading: The remaining years of the 2020s will be wild for AI.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“David Ferrucci, CEO of Elemental Cognition, and director of IBM’s successful Watson Jeopardy effort, worried that current systems were “ultimately unsatisfied”, and like me called for hybrid approaches that combine neural networks with reasoning and structured representations.”
Our reading: AI should pursue hybrid approaches combining neural networks with reasoning and structured representations.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“The trouble is, those guardrails don’t come for free; the system is still shallow; ChatGPT doesn’t really know what it is it is guarding against; it mostly seems instead to be just looking for keywords, which is why it is prone to generating nonsense like this:”
Our reading: ChatGPT's guardrails don't really understand what they guard against and appear to rely mostly on keywords, which makes ChatGPT prone to producing nonsense.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“There’s another problem, too, arguably even bigger concern in the context of search engines: hallucination.”
Our reading: For search engines, hallucination is a further problem for LLM-based systems and arguably an even bigger concern than the shallowness, or discomprehension, he described just before.
No horizon
Scope: in the context of search engines
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“But until those bits all fit together in a way we can trust, I prefer to borrow a memorable phrase from Ariana Grande: Thank U, Next.”
Our reading: Until the pieces of LLM-based search fit together in a way that can be trusted, he personally prefers to pass on it ('Thank U, Next').
No horizon
Qualifier: until those bits all fit together in a way we can trust
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“In the quest for a newfangled search I suspect we are once again in for a rough ride.”
Our reading: Developing LLM-based search will probably be a difficult, drawn-out process, as with driverless cars.
No horizon