Unresolved claims, page 19 of 20

Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.

Unresolved claims (1958)

The record was checked, and the evidence available by the cutoff does not settle it. Forecasts whose deadline has not arrived are listed apart, as not yet due.

Page 19 of 20: claims 1801 to 1900 of 1958, ordered by the date first said, newest first. To find a claim by its words, use search.

Assessment: UnresolvedSaid 2023-03-02
Gary Marcus
“I presented this example of how even the latest version of Bing, with its new guardrails, could still be manipulated to be useful to troll farms:”
Assessment: UnresolvedSaid 2023-03-02
Gary Marcus
“Meanwhile, within a few hours, a 17-year-old high-school student had largely1 replicated what I was saying, and did so with a simpler, more elegant jailbreak prompt, zero prior context required2:”
Assessment: UnresolvedSaid 2023-02-28
Gary Marcus
“What’s even more disturbing is that Bing makes it look like the false narrative that it generates is referenced.”
Assessment: UnresolvedSaid 2023-02-22
Gary Marcus
“Few people understand how they work; many people anthropomorphizing those chatbots, attributing to them real intelligence and emotion.”
Assessment: UnresolvedSaid 2023-02-22
Gary Marcus
“But no known technology can reliably align a chatbot in a persistent way to a human’s emotional needs.”
Assessment: UnresolvedSaid 2023-02-22
Gary Marcus
“with the new generation of chatbots, we will see addictions no less potent.”
Assessment: UnresolvedSaid 2023-02-20
Gary Marcus
“Right now, as I have said before, the AI is basically the Wild West; anyone can post any chatbot they want.”
Assessment: UnresolvedSaid 2023-02-20
Gary Marcus
“This would be very bad news, not just in terms of human and economic costs (which would mean more underpaid layers doing awful work) but also in terms of trustworthiness, because it would be mean that we have zero guarantee that any new iteration of a large language model is going to safe.”
Assessment: UnresolvedSaid 2023-02-20
Gary Marcus
“Anyone who watched the last week unfold will realize that the new Bing has (or had1) a tendency to get really wild, from declaring a love that it didn’t really have to encouraging people to get divorced to blackmailing them to teaching people how to commit crimes, and so on.”
Assessment: UnresolvedSaid 2023-02-17
Gary Marcus
“The problems in physical and psychological reasoning that we pointed in the Fall of 2020 very much remain.”
Assessment: UnresolvedSaid 2023-02-17
Gary Marcus
“For one thing, what we publish often gets incorporated into the training; perhaps in company benchmarks as well.”
Assessment: UnresolvedSaid 2023-02-17
Gary Marcus
“But In practice, that isn’t going to happen.”
Assessment: UnresolvedSaid 2023-02-17
Gary Marcus
“However, just in terms of percentage right vs. wrong, the improvement was slight; It now got 50% right, 40% wrong, and 10% borderline (marginally better than the 45% right before).”
Assessment: UnresolvedSaid 2023-02-16
Gary Marcus
“each reflect the fact that you cannot build AI in the real world from Big Data and deep learning alone.”
Assessment: UnresolvedSaid 2023-02-16
Gary Marcus
“There was a broken image link, and I went so fast I neglected to mention the personally salient fact that Bing had lied about me, claiming that I believe that “Bing’s AI was more accurate and reliable than Bard”, when I never said any such thing.”
Assessment: UnresolvedSaid 2023-02-16
Gary Marcus
“Microsoft can (at least to a first approximation) do whatever the hell it wants with Bing.”
Assessment: UnresolvedSaid 2023-02-16
Gary Marcus
“There are almost no regulations here; we are living in the Wild AI West.”
Assessment: UnresolvedSaid 2023-02-16
Ed Zitron
“The result will be genuine human suffering as a result of dressing a fallible, experimental intelligence up as an authoritative source of information. People will be inspired to harm themselves or others, all while their ideas and likenesses are counterfeited and monetized, all so that wealthy executives can tell the world - and Wall Street - that their companies are “innovative.””
Assessment: UnresolvedSaid 2023-02-11
Gary Marcus
“And, bonus, there is little if any government regulation in place to do much about this.”
Assessment: UnresolvedSaid 2023-02-09
Gary Marcus
“I do actually think that these problems will eventually get ironed out, possibly after some important fresh discoveries are made, but whether they get ironed out soon is another matter entirely.”
Assessment: UnresolvedSaid 2023-02-09
Ed Zitron
“These moves are unquestionably hurting Google, to the point that Microsoft’s Bing (paired with OpenAI’s ChatGPT), has gained major headlines for providing the service that everybody wished Google would.”
Assessment: UnresolvedSaid 2023-02-06
Ed Zitron
“Mark Zuckerberg was wrong about the metaverse, Marc Andreessen and Chris Dixon were wrong about Web3, David Sacks was wrong about just about everything, and Google has obfuscated the purpose of their search engine so badly that they have had to beg their co-founders to return to help fight OpenAI’s ChatGPT, a product from a company of a few hundred people which just got a $14 billion investment from their biggest competitor (just after said competitor laid off 10,000 people).”
Assessment: UnresolvedSaid 2023-02-03
Gary Marcus
“Our new systems work better, but they are still superficial, fine for lab demos but often at a loss in the real world.”
Assessment: UnresolvedSaid 2023-02-03
Gary Marcus
“Deep learning has undeniably made amazing progress, on other dimensions, but the fact is it hasn’t solved the core problems that many of us thought were essential 30 years ago.”
Assessment: UnresolvedSaid 2023-02-03
Gary Marcus
“Melanie Mitchell’s book, Bender et al’s “Stochastic Parrots”; all of these more recent criticisms of deep learning help establish that some of the basic issues that were known long ago, with predecessors to current models, still remain even today.”
Assessment: UnresolvedSaid 2023-02-03
Gary Marcus
“Not sure how much has changed in that regard, either, except that the hacks that were once written by graduate students are now farmed out to sweatshops, for a new form of handcrafting:”
Assessment: UnresolvedSaid 2023-02-03
Gary Marcus
“Turns out you don’t need to solve them to make beautiful pictures, or decent if error-filled conversation, but anyone who thinks we can get to AGI without solving abstraction, compositionality, and the type-token distinction is kidding themselves.”
Assessment: UnresolvedSaid 2023-02-03
Gary Marcus
“The measures are new, the model is new, but the stumbling block remains the same.”
Assessment: UnresolvedSaid 2023-02-02
Gary Marcus
“And I still attribute those problems to a lack of world models that represent the ways in which things unfold over time. I still see this as the central flaw in large language models.”
Assessment: UnresolvedSaid 2023-02-02
Gary Marcus
“But more than three years on, it’s still—even after a new update a few days ago focused around math—having lot of problems with easy word problems like this:”
Assessment: UnresolvedSaid 2023-02-02
Gary Marcus
“I said the problem wasn’t math per se, but that large language models didn’t represent robust world models of how events fold over time; I stand by this 100% even today.”
Assessment: UnresolvedSaid 2023-01-31
Gary Marcus
“Anyone who thinks about how AI is used should know about it—especially as we are about to head into an era of ubiquitous yet fallible AI.”
Assessment: UnresolvedSaid 2023-01-22
Gary Marcus
“Machines will just get more and more reliable and more general.”
Assessment: UnresolvedSaid 2023-01-22
Gary Marcus
“Neither one of us believes for a minute that large language models are remotely close to artificial general intelligence, and what’s more, we mostly agree about what’s missing: semantics, reasoning, common sense, theory of mind and so on.”
Assessment: UnresolvedSaid 2023-01-22
Gary Marcus
“I think that in achieving AGI we will have no choice but to find ways of integrating whatever we can get out of deep learning with explicit knowledge and explicit cognitive models.”
Assessment: UnresolvedSaid 2023-01-22
Gary Marcus
“A decent fraction of the AI community has already recognized that the primary forte of large language models is in generating bullshit; many also see that the current path to autonomous vehicles is not making the progress that some expected.”
Assessment: UnresolvedSaid 2023-01-22
Gary Marcus
“In a recent Wired article, I predicted that 2023 see the first death attributable to large language models, and I stand by that prediction.”
Assessment: UnresolvedSaid 2023-01-22
Gary Marcus
“The neurosymbolic AI community is growing fast; conferences that used to be dozens are now thousands.”
Assessment: UnresolvedSaid 2023-01-14
Gary Marcus
“ChatGPT simply cannot reason well enough to to do this; as I have documented in previous essays, it has demonstrably misunderstood physics, biology, psychology, medicine, and so forth.”
Assessment: UnresolvedSaid 2023-01-14
Gary Marcus
“It is not contributing genuine ideas, nor coming up with carefully controlled experimental designs, nor summing up prior literature in thoughtful ways.”
Assessment: UnresolvedSaid 2023-01-14
Gary Marcus
“We are going to see a lot of this – misuses of ChatGPT in which people trust it, and even trumpet it—when it’s flat-out wrong.”
Assessment: UnresolvedSaid 2023-01-11
Gary Marcus
“That’s a lot, and not the sort of thing that a rocketship startup would typically agree to.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“For another, although OpenAI may be tracking errors like these for internal purposes, the scientific community has no access to whatever they have collected, which leaves it in the lurch.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“For one thing, although many people seem to have heard of a few of these errors (e.g. from reports in the media), few realize how pervasive or broad in scope they are; indeed a recent Twitter thread that claimed falsely that these problems had been resolved got a surprising amount of traction.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“ChatGPT made all the same kinds of mistakes that its predecessors did.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“It is therefore impossible for external scientists to determine to what extent any specific correct answer merely reflects some very similar examples in the training data.”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“ChatGPT couldn’t, for example, reliably count to four (figure 1) or do one-digit arithmetic in the context of simple word problem (figure 2).”
Assessment: UnresolvedSaid 2023-01-09
Gary Marcus
“In our opinion, this was inevitable, in a system that has been trained to optimize the goal of producing one plausible-sounding word after another rather than actually engage with the meaning of language.”
Assessment: UnresolvedSaid 2022-12-31
Gary Marcus
“(Spoiler alert: they’re not, at least not yet.)”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“But as they do so, as they glom stuff together, something often gets lost in translation: which bits of text do and do not truly belong together.”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“Once again, the result was fluent prose, and once again, hallucination ruled:”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“As it happens, large language models are also wicked hard to update, typically requiring full retraining, sometimes taking weeks or months.”
Assessment: UnresolvedSaid 2022-12-30
Gary Marcus
“Which means for example that ChatGPT, released in November, is so old it doesn’t know who owns Twitter:”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Without external aids it won’t be able beat Meta’s Cicero in Diplomacy; it won’t be able to drive a car reliably; it won’t be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“GPT-4 will be trained on considerably more, a significant fraction of the internet as a whole.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Its natural language output still won’t be something that one can reliably hook up to downstream programs; it won’t be something, for example, that you can simply and directly hook up to a database or virtual assistant, with predictable results.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“GPT-4 may be better at faking term papers, but if it follows the same playbook as it predecessors, it still won’t really understand the world, and seams will eventually show.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“GPT-4 will not have reliable models of the things that it talks about that are accessible to external programmers in a way that reliably feeds downstream processes.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“As I told NYT columnist Farhad Manjoo, ChatGPT, like earlier, related systems is “still not reliable, still doesn’t understand the physical world, still doesn’t understand the psychological world and still hallucinates.””
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Although GPT-4 will definitely seem smarter than its predecessors, its internal architecture remains problematic.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
“Which is to say it makes stuff up, regularly; a fair bit of what it is says simply isn’t true.”
Assessment: UnresolvedSaid 2022-12-25
Gary Marcus
““Scaling” alone—building bigger and models until they absorb the entire internet — will prove useful, but only to a point.”
Assessment: UnresolvedSaid 2022-12-10
Gary Marcus
“they cost almost nothing to operate, and so they are on a path to reducing the cost of generating disinformation to zero.”
Assessment: UnresolvedSaid 2022-12-10
Gary Marcus
“they can easily be automated to generate misinformation at unprecedented scale.”
Assessment: UnresolvedSaid 2022-12-10
Gary Marcus
“If the website is flooded with worthless code examples, programmers will no longer go there, its database of over 30 million questions and answers will become untrustworthy, and the 14 year old website will die.”
Assessment: UnresolvedSaid 2022-12-01
Gary Marcus
“GPT”s heavy use of a technique called embeddings makes it really good at substituting synonyms and more broadly related phrases, but the same tendency towards substitution often lead it astray.”
Assessment: UnresolvedSaid 2022-12-01
Gary Marcus
“When GPT sounds plausible, it is because every paraphrased bit that it pastes together is grounded in something that actual humans said, and there is often some vague (but often irrelevant) relationship between..”
Assessment: UnresolvedSaid 2022-12-01
Gary Marcus
“Almost everything it says is fluent and at least vaguely plausible.”
Assessment: UnresolvedSaid 2022-11-25
Gary Marcus
“As it turns out, the architecture of Cicero differs profoundly from most of what’s been talked about in recent years in AI.”
Assessment: UnresolvedSaid 2022-11-22
Gary Marcus
“See also my essay on Google’s Palm-SayCan, for what might happen if this tech were embedded in robots, which would be even more risky.”
Assessment: UnresolvedSaid 2022-11-22
Gary Marcus
“if Alexa could converse freely, user expectations would go through the roof, and mostly be unmeetable.”
Assessment: UnresolvedSaid 2022-11-22
Gary Marcus
“Turning LLMs into a product that controls your home and talks to you in a way that would be reliable enough to use at scale in millions of homes is still a long, long way away.”
Assessment: UnresolvedSaid 2022-11-16
Gary Marcus
“Just like every other large language model I have seen.”
Assessment: UnresolvedSaid 2022-11-16
Gary Marcus
“High school students will love it, and use it to fool and intimidate (some of) their teachers.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“The problem is about to get much, much worse.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“As the pace of machine-generated misinformation picks up, Twitter’s existing effort, Community Notes (aka Birdwatch), which is mostly done manually, by humans, is going to get left in the dust.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“Large language models are without equal at generating misinformation, but they suck at detecting it.”
Assessment: UnresolvedSaid 2022-11-09
Gary Marcus
“Scaling won’t solve this.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“Scaling maximalism is an interesting hypothesis, but I stand by my prediction that it won’t get us to AGI.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“Any one of them would mean we need a real paradigm shift, if we are to get to AGI.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“His hypothesis, now generally called scaling maximalism, remains extremely popular, in no small part because bigger and bigger models have indeed continued to do ever more impressive things.”
Assessment: UnresolvedSaid 2022-10-29
Gary Marcus
“As I have long argued, capturing this without cognitive models and common sense is really hard.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“We know how to build machines that do math (augmentation); we don’t know how to build machines that can read tax codes (replacement).”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“We can’t really build humanoid domestic robots without general intelligence, and we don’t yet have any serious candidates.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Hedging with probably makes it sounds like we think there is a viable competitor out there for human general intelligence in the AI world. There isn’t.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Every truck I know is still driven by a person (except in some limited-use test pilots), and no radiologists have yet been replaced.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Human children are constantly learning new things, often from tiny amounts of data. There is literally nothing like that in the AI world.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Eleven years later, comprehension is still lacking (see any of my recent essays here) and very few if any jobs have actually been replaced by AI.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“There is no evidence whatsoever that what has gotten us, sort of, into the game of reading scans is going to bring us all the way into the promised land of a radiologist in an app any time soon.”
Assessment: UnresolvedSaid 2022-10-16
Gary Marcus
“Well, no. Augmentation is way easier, because you don’t have to solve the whole job.”
Assessment: UnresolvedSaid 2022-10-02
Gary Marcus
“It’s also, as ever, stuck in the liminal space between superintelligence and super-lost.”
Assessment: UnresolvedSaid 2022-10-02
Gary Marcus
“This is the uncanny valley we are in, and one that we occupy for a while.”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“We in AI have a long long way to go. Data and compute alone won’t be enough.”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“Musk barely even acknowledged the problem, and certainly didn’t lay out any sort of credible strategy for attacking it..”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“The reality is that AI needs genuine, paradigm-shift level innovation. Simply building ever bigger neural networks won’t cut it.”
Assessment: UnresolvedSaid 2022-10-01
Gary Marcus
“Second, there was very little vision for how Tesla would build the cognitive part of the AI they will need, beyond the basics of motor control (which Boston Dynamics already does so well), nor much recognition about why robotics is so hard in the real world.”
Assessment: UnresolvedSaid 2022-09-29
Gary Marcus
“Musk is almost never right about deadlines.”
Assessment: UnresolvedSaid 2022-09-23
Gary Marcus
“Current neural nets lack the cognitive depth and robustness that would be necessary to support reliable L5 self-driving form vision alone.”
Assessment: UnresolvedSaid 2022-09-23
Gary Marcus
“The problem for the neural network community is not that I have changed my goal posts, it’s that my 2012 concerns still stand.”
Assessment: UnresolvedSaid 2022-09-23
Gary Marcus
“The argument I made there is that … systems like DALL-E don’t “understand what wheels are—or what they are for”, because (as I discovered on Sunday) they systematically fail to properly illustrate requests like these:”