Unresolved claims, page 23 of 24

Claims by Gary Marcus and Ed Zitron, as of 2026-11-11.

Unresolved claims (2,331)

We checked, and the evidence so far doesn’t settle it. Predictions whose deadline hasn’t come yet are under Not yet due.

Claims 2,201 to 2,300 of 2,331, newest first. Page 23 of 24.

Assessment: UnresolvedSaid 2023-01-14
Gary Marcus
“ChatGPT simply cannot reason well enough to to do this; as I have documented in previous essays, it has demonstrably misunderstood physics, biology, psychology, medicine, and so forth.”
Assessment: UnresolvedSaid 2023-01-14
Gary Marcus
“It is not contributing genuine ideas, nor coming up with carefully controlled experimental designs, nor summing up prior literature in thoughtful ways.”
Assessment: UnresolvedSaid 2023-01-14
Gary Marcus
“We are going to see a lot of this – misuses of ChatGPT in which people trust it, and even trumpet it—when it’s flat-out wrong.”
Assessment: UnresolvedSaid 2023-01-11
Gary Marcus
“That’s a lot, and not the sort of thing that a rocketship startup would typically agree to.”
Assessment: UnresolvedSaid 2023-01-11
Gary Marcus
“Meanwhile, large language models themselves are quickly becoming commodities; it’s not clear that there is any unique, protectable intellectual property here.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“For another, although OpenAI may be tracking errors like these for internal purposes, the scientific community has no access to whatever they have collected, which leaves it in the lurch.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“For one thing, although many people seem to have heard of a few of these errors (e.g. from reports in the media), few realize how pervasive or broad in scope they are; indeed a recent Twitter thread that claimed falsely that these problems had been resolved got a surprising amount of traction.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“ChatGPT made all the same kinds of mistakes that its predecessors did.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“It is therefore impossible for external scientists to determine to what extent any specific correct answer merely reflects some very similar examples in the training data.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“ChatGPT couldn’t, for example, reliably count to four (figure 1) or do one-digit arithmetic in the context of simple word problem (figure 2).”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“In our opinion, this was inevitable, in a system that has been trained to optimize the goal of producing one plausible-sounding word after another rather than actually engage with the meaning of language.”
Assessment: UnresolvedSaid 2022-12-31
Gary Marcus
“(Spoiler alert: they’re not, at least not yet.)”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“But as they do so, as they glom stuff together, something often gets lost in translation: which bits of text do and do not truly belong together.”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“Once again, the result was fluent prose, and once again, hallucination ruled:”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“As it happens, large language models are also wicked hard to update, typically requiring full retraining, sometimes taking weeks or months.”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“Which means for example that ChatGPT, released in November, is so old it doesn’t know who owns Twitter:”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Without external aids it won’t be able beat Meta’s Cicero in Diplomacy; it won’t be able to drive a car reliably; it won’t be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“GPT-4 will be trained on considerably more, a significant fraction of the internet as a whole.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Its natural language output still won’t be something that one can reliably hook up to downstream programs; it won’t be something, for example, that you can simply and directly hook up to a database or virtual assistant, with predictable results.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“GPT-4 may be better at faking term papers, but if it follows the same playbook as it predecessors, it still won’t really understand the world, and seams will eventually show.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“GPT-4 will not have reliable models of the things that it talks about that are accessible to external programmers in a way that reliably feeds downstream processes.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“As I told NYT columnist Farhad Manjoo, ChatGPT, like earlier, related systems is “still not reliable, still doesn’t understand the physical world, still doesn’t understand the psychological world and still hallucinates.””
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Although GPT-4 will definitely seem smarter than its predecessors, its internal architecture remains problematic.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Which is to say it makes stuff up, regularly; a fair bit of what it is says simply isn’t true.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“When it comes out, it will totally eclipse ChatGPT; it’s safe bet that even more people will be talking about it..”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
““Scaling” alone—building bigger and models until they absorb the entire internet — will prove useful, but only to a point.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Trustworthy, general artificial intelligence, aligned with human values, will come, when it does, from systems that are more structured, with more built-in knowledge, and will incorporate at least some degree of explicit tools for reasoning and planning, as well as explicit it knowledge, that are lacking in systems like GPT.”
Assessment: UnresolvedSaid 2022-12-10
Gary Marcus
“they cost almost nothing to operate, and so they are on a path to reducing the cost of generating disinformation to zero.”
Assessment: UnresolvedSaid 2022-12-10
Gary Marcus
“they can easily be automated to generate misinformation at unprecedented scale.”
Assessment: UnresolvedSaid 2022-12-10
Gary Marcus
“If the website is flooded with worthless code examples, programmers will no longer go there, its database of over 30 million questions and answers will become untrustworthy, and the 14 year old website will die.”
Assessment: UnresolvedSaid 2022-12-01
Gary Marcus
“GPT”s heavy use of a technique called embeddings makes it really good at substituting synonyms and more broadly related phrases, but the same tendency towards substitution often lead it astray.”
Assessment: UnresolvedSaid 2022-12-01
Gary Marcus
“When GPT sounds plausible, it is because every paraphrased bit that it pastes together is grounded in something that actual humans said, and there is often some vague (but often irrelevant) relationship between..”
Assessment: UnresolvedSaid 2022-12-01
Gary Marcus
“Almost everything it says is fluent and at least vaguely plausible.”
Assessment: UnresolvedSaid 2022-11-25
Gary Marcus
“As it turns out, the architecture of Cicero differs profoundly from most of what’s been talked about in recent years in AI.”
Assessment: UnresolvedSaid 2022-11-22
Gary Marcus
“See also my essay on Google’s Palm-SayCan, for what might happen if this tech were embedded in robots, which would be even more risky.”
Assessment: UnresolvedSaid 2022-11-22
Gary Marcus
“if Alexa could converse freely, user expectations would go through the roof, and mostly be unmeetable.”
Assessment: UnresolvedSaid 2022-11-22
Gary Marcus
“Turning LLMs into a product that controls your home and talks to you in a way that would be reliable enough to use at scale in millions of homes is still a long, long way away.”
Assessment: UnresolvedSaid 2022-11-16
Gary Marcus
“Just like every other large language model I have seen.”
Assessment: UnresolvedSaid 2022-11-16
Gary Marcus
“High school students will love it, and use it to fool and intimidate (some of) their teachers.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“The problem is about to get much, much worse.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“As the pace of machine-generated misinformation picks up, Twitter’s existing effort, Community Notes (aka Birdwatch), which is mostly done manually, by humans, is going to get left in the dust.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“Large language models are without equal at generating misinformation, but they suck at detecting it.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“Scaling won’t solve this.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“Scaling maximalism is an interesting hypothesis, but I stand by my prediction that it won’t get us to AGI.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“Any one of them would mean we need a real paradigm shift, if we are to get to AGI.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“His hypothesis, now generally called scaling maximalism, remains extremely popular, in no small part because bigger and bigger models have indeed continued to do ever more impressive things.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“As I have long argued, capturing this without cognitive models and common sense is really hard.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“We know how to build machines that do math (augmentation); we don’t know how to build machines that can read tax codes (replacement).”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“We can’t really build humanoid domestic robots without general intelligence, and we don’t yet have any serious candidates.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Hedging with probably makes it sounds like we think there is a viable competitor out there for human general intelligence in the AI world. There isn’t.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Every truck I know is still driven by a person (except in some limited-use test pilots), and no radiologists have yet been replaced.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Human children are constantly learning new things, often from tiny amounts of data. There is literally nothing like that in the AI world.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Eleven years later, comprehension is still lacking (see any of my recent essays here) and very few if any jobs have actually been replaced by AI.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“There is no evidence whatsoever that what has gotten us, sort of, into the game of reading scans is going to bring us all the way into the promised land of a radiologist in an app any time soon.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“The answer is that at present we have no viable option; AlphaStar and LLMs would surely be unsafe and inadequate; and we don’t yet have anything markedly better.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Well, no. Augmentation is way easier, because you don’t have to solve the whole job.”
Assessment: UnresolvedSaid 2022-10-02
Gary Marcus
“It’s also, as ever, stuck in the liminal space between superintelligence and super-lost.”
Assessment: UnresolvedSaid 2022-10-02
Gary Marcus
“This is the uncanny valley we are in, and one that we occupy for a while.”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“We in AI have a long long way to go. Data and compute alone won’t be enough.”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“Musk barely even acknowledged the problem, and certainly didn’t lay out any sort of credible strategy for attacking it..”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“The reality is that AI needs genuine, paradigm-shift level innovation. Simply building ever bigger neural networks won’t cut it.”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“Second, there was very little vision for how Tesla would build the cognitive part of the AI they will need, beyond the basics of motor control (which Boston Dynamics already does so well), nor much recognition about why robotics is so hard in the real world.”
Assessment: UnresolvedSaid 2022-09-29
Gary Marcus
“Musk is almost never right about deadlines.”
Assessment: UnresolvedSaid 2022-09-23
Gary Marcus
“Current neural nets lack the cognitive depth and robustness that would be necessary to support reliable L5 self-driving form vision alone.”
Assessment: UnresolvedSaid 2022-09-23
Gary Marcus
“The problem for the neural network community is not that I have changed my goal posts, it’s that my 2012 concerns still stand.”
Assessment: UnresolvedSaid 2022-09-23
Gary Marcus
“The argument I made there is that … systems like DALL-E don’t “understand what wheels are—or what they are for”, because (as I discovered on Sunday) they systematically fail to properly illustrate requests like these:”
Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“On #1, the answer is a clear yes; only highly trained human artists could do better.”
Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“Nine more tries, and only one very marginal success (top right corner):”
Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“Even “draw a white bicycle with green wheels”, which focuses purely on part-whole relationships without function or complex syntax, is problematic:”
Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“Negation is, as ever, a problem.”
Assessment: UnresolvedSaid 2022-09-14
Gary Marcus
“Google cloaks its current work the trappings of science (article preprints with citations and graphs and so forth) but when it comes to community inspection, they’ve systematically opted out.”
Assessment: UnresolvedSaid 2022-09-14
Gary Marcus
“Notwithstanding the rumors about Google Imagen, nobody has yet publicly demonstrated a machine that can relate the meanings of sentences to their parts the way a five-year-old child can.”
Assessment: UnresolvedSaid 2022-09-14
Gary Marcus
“If another AI winter does comes, it not be because AI is impossible, but because AI hype exceeds reality. The only cure for that is truth in advertising.”
Assessment: UnresolvedSaid 2022-09-14
Gary Marcus
“But many people have claimed over the years to solve the problem, and none of those systems have proven to be reliable; every proposed solution has been like Clever Hans, working in dim light, leveraging large databases to some degree, but falling apart upon careful inspection; they might get 60% on some task, but they never really master it.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Building a robot on top of language system that has such little comprehension of the world can’t be a recipe for success.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Cool demo, yes. Real world product, anytime soon? Fuggedaboutit.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Big data won’t be enough to solve the problems of robotics”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Facebook M was canceled, and 7 years later nobody yet can buy a super-smart (or even decently smart) robot at any price.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Google makes a big deal about how the system is to do some degree interpretable, but there is no obvious way to integrate large language models with the kind of formal formal verification methods that are now routine in the design of microprocessors, USB drivers and large aircraft.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“Indeed systems like GPT-3 are spectacular at memorization, but many onlookers recognize them for what they are, stochastic imitations of their input, with very little real coherence and very little comprehension.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“The thing is, current AI systems that (roughly speaking) do a lot of cutting and pasting without those grammar rules and trees aren’t really all that close to understanding human language.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“We don’t know how kids do it; we do know that what we are doing with machines currently isn’t really working.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“All normal children manage to acquire language in this sense; no existing machine does.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“In our view, inadequate attention to these three factors has serious consequences, including:”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“here, DALL-E mini struggles to accurately represent the implicit action for the woman, capturing the apple but not reliably capturing the eating, often sticks an apple in the man’s mouth rather than the sandwich, and in most cases misses the implication that the man and woman are supposed to be eating different things.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“The fact that its word sequence predictions are often so good tempts many into ascribing meaning, but this is more a reflection of the tremendous size of the input corpus than of a system that genuinely maps sentence onto reference and cognitive models.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Similarly, consider pronouns like “himself”: DALL-E does not do well with sentences like “The man is painting a picture of himself”.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Large language models don’t produce such cognitive models, at least not in a way that anybody has been able to make reliable use of.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Compositionality has long been a central concept in linguistics and philosophy (even Aristotle acknowledged it), yet so-called foundation models (large language models, GPT-3, BERT, deep learning approaches) sidestep it.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“(a) the tendency of large language models to lose coherence over time, drifting into “empty” language with no clear connection to reality;”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“(b) the difficulty of large language models in distinguishing truth from falsehoods;”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“But the interesting thing to note here is that we humans can choose to interpret language compositionally, or not—this degree of cognitive flexibility is really quite impressive, but lacking in current large language models.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Language is about a connection between words (or sentence) and the world; the sequences of words that large language models utter lack connection to the external world.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“In such cases, DALL-E Mini typically generates images of bowls of cucumbers, and only cucumbers, leaving out the tomatoes altogether (where a human might reasonably expect there to be at least two, given the plural).”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“The “states” that such systems visit are pretty, but lack reference, and lack ties to dynamically updated cognitive models.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Unless the field can find a way to tightly integrate large language models with cognitive models and reference, we expect that the field will eventually hit a dead end, particularly in the understanding long discourses, and series of events unfolding over time, and that the problem of misinformation will remain unsolved.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“(c) the struggle in these models to avoid perpetuating bias and toxic speech.”
Assessment: UnresolvedSaid 2022-06-15
Gary Marcus
“Does that mean that society should give up on building driverless cars? Absolutely not. In the long run, they will save lives, and also give many people, such the blind, elderly, and disabled, have far more autonomy over their own lives.”
Assessment: UnresolvedSaid 2022-06-15
Gary Marcus
“As far as I can tell, nobody has a real solution to the outlier problem of coping with unusual circumstances; without it, full self driving will remain a pipe dream.”
Assessment: UnresolvedSaid 2022-06-12
Gary Marcus
“We in the AI community have our differences, but pretty much all of find the notion that LaMDA might be sentient completely ridiculous.”