Unresolved claims, page 20 of 20

Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.

Unresolved claims (1958)

The record was checked, and the evidence available by the cutoff does not settle it. Forecasts whose deadline has not arrived are listed apart, as not yet due.

Page 20 of 20: claims 1901 to 1958 of 1958, ordered by the date first said, newest first. To find a claim by its words, use search.

Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“On #1, the answer is a clear yes; only highly trained human artists could do better.”
Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“Nine more tries, and only one very marginal success (top right corner):”
Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“Even “draw a white bicycle with green wheels”, which focuses purely on part-whole relationships without function or complex syntax, is problematic:”
Assessment: UnresolvedSaid 2022-09-18
Gary Marcus
“Negation is, as ever, a problem.”
Assessment: UnresolvedSaid 2022-09-14
Gary Marcus
“Google cloaks its current work the trappings of science (article preprints with citations and graphs and so forth) but when it comes to community inspection, they’ve systematically opted out.”
Assessment: UnresolvedSaid 2022-09-14
Gary Marcus
“Notwithstanding the rumors about Google Imagen, nobody has yet publicly demonstrated a machine that can relate the meanings of sentences to their parts the way a five-year-old child can.”
Assessment: UnresolvedSaid 2022-09-14
Gary Marcus
“But many people have claimed over the years to solve the problem, and none of those systems have proven to be reliable; every proposed solution has been like Clever Hans, working in dim light, leveraging large databases to some degree, but falling apart upon careful inspection; they might get 60% on some task, but they never really master it.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Building a robot on top of language system that has such little comprehension of the world can’t be a recipe for success.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Cool demo, yes. Real world product, anytime soon? Fuggedaboutit.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Big data won’t be enough to solve the problems of robotics”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Facebook M was canceled, and 7 years later nobody yet can buy a super-smart (or even decently smart) robot at any price.”
Assessment: UnresolvedSaid 2022-08-18
Gary Marcus
“Google makes a big deal about how the system is to do some degree interpretable, but there is no obvious way to integrate large language models with the kind of formal formal verification methods that are now routine in the design of microprocessors, USB drivers and large aircraft.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“Indeed systems like GPT-3 are spectacular at memorization, but many onlookers recognize them for what they are, stochastic imitations of their input, with very little real coherence and very little comprehension.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“The thing is, current AI systems that (roughly speaking) do a lot of cutting and pasting without those grammar rules and trees aren’t really all that close to understanding human language.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“We don’t know how kids do it; we do know that what we are doing with machines currently isn’t really working.”
Assessment: UnresolvedSaid 2022-07-20
Gary Marcus
“All normal children manage to acquire language in this sense; no existing machine does.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“In our view, inadequate attention to these three factors has serious consequences, including:”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“here, DALL-E mini struggles to accurately represent the implicit action for the woman, capturing the apple but not reliably capturing the eating, often sticks an apple in the man’s mouth rather than the sandwich, and in most cases misses the implication that the man and woman are supposed to be eating different things.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“The fact that its word sequence predictions are often so good tempts many into ascribing meaning, but this is more a reflection of the tremendous size of the input corpus than of a system that genuinely maps sentence onto reference and cognitive models.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Similarly, consider pronouns like “himself”: DALL-E does not do well with sentences like “The man is painting a picture of himself”.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Compositionality has long been a central concept in linguistics and philosophy (even Aristotle acknowledged it), yet so-called foundation models (large language models, GPT-3, BERT, deep learning approaches) sidestep it.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“(a) the tendency of large language models to lose coherence over time, drifting into “empty” language with no clear connection to reality;”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“(b) the difficulty of large language models in distinguishing truth from falsehoods;”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“But the interesting thing to note here is that we humans can choose to interpret language compositionally, or not—this degree of cognitive flexibility is really quite impressive, but lacking in current large language models.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Language is about a connection between words (or sentence) and the world; the sequences of words that large language models utter lack connection to the external world.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“In such cases, DALL-E Mini typically generates images of bowls of cucumbers, and only cucumbers, leaving out the tomatoes altogether (where a human might reasonably expect there to be at least two, given the plural).”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“Unless the field can find a way to tightly integrate large language models with cognitive models and reference, we expect that the field will eventually hit a dead end, particularly in the understanding long discourses, and series of events unfolding over time, and that the problem of misinformation will remain unsolved.”
Assessment: UnresolvedSaid 2022-06-22
Gary Marcus
“(c) the struggle in these models to avoid perpetuating bias and toxic speech.”
Assessment: UnresolvedSaid 2022-06-15
Gary Marcus
“As far as I can tell, nobody has a real solution to the outlier problem of coping with unusual circumstances; without it, full self driving will remain a pipe dream.”
Assessment: UnresolvedSaid 2022-06-12
Gary Marcus
“We in the AI community have our differences, but pretty much all of find the notion that LaMDA might be sentient completely ridiculous.”
Assessment: UnresolvedSaid 2022-06-12
Gary Marcus
“The public knows the test of course, but the AI community wishes it would go away; we all know that beating that test isn’t meaningful.”
Assessment: UnresolvedSaid 2022-06-11
Gary Marcus
“👉an explicit type/token distinction will be crucial”
Assessment: UnresolvedSaid 2022-06-11
Gary Marcus
“👉explicit cognitive models will be crucial”
Assessment: UnresolvedSaid 2022-06-11
Gary Marcus
“👉operations over variables (including storing, retrieving and comparing values) will be crucial”
Assessment: UnresolvedSaid 2022-06-07
Gary Marcus
“Large language models are (unless hybridized with symbolic systems, a la Jurassic X) are, on their own, inherently weak at understanding the physical and psychological world.”
Assessment: UnresolvedSaid 2022-06-07
Gary Marcus
“Interestingly, as Ernie has just discovered, across the full set of our Technology Review questions the difference from GPT-3’s earlier version to the current version (InstructGPT) is far more modest than one might have expected: fewer outright wacky answers, some old answers now correct, but some backsliding, too.”
Assessment: UnresolvedSaid 2022-06-07
Gary Marcus
“The core claims that Ernie and I made in Technology Review still all, so far as I can tell, hold true, nearly two years and many even larger models later. Yes, some of the specific examples we pointed out have been remedied, but the general problems have not.”
Assessment: UnresolvedSaid 2022-06-07
Gary Marcus
“But they still don’t convey genuine comprehension, and so they are still very easy for Ernie and me (or anyone else who cares to try) to break.”
Assessment: UnresolvedSaid 2022-06-07
Gary Marcus
“I am willing to bet SlateStarCodex now (terms to be negotiated) that if OpenAI gives us unrestricted access to GPT-4, whenever that is released, and assuming that is basically the same architecture but with more data, that within a day of playing around with it, Ernie and I will still be able find lots of examples of failures in physical reasoning, temporal reasoning, causal reasoning, and so forth.”
Assessment: UnresolvedSaid 2022-06-07
Gary Marcus
“The improvements, such as they are, come, primarily because the newer models have larger and larger sets of data about how human beings use word sequences, and bigger word sequences are certainly helpful for pattern matching machines.”
Assessment: UnresolvedSaid 2022-05-31
Gary Marcus
“The fourth thing to realize is that we, and by this I mean humanity as a whole, still don’t really have an adequate methodology for building complex cognitive systems.”
Assessment: UnresolvedSaid 2022-05-31
Gary Marcus
“I don’t know what you have planned for Optimus, but I can promise you that the AGI that you would need for a general-purpose domestic robot (where every home is different, and each poses its own safety risks) is way beyond what you would need for a car that drives on roads that are more or less engineered the same way from one town to the next.”
Assessment: UnresolvedSaid 2022-05-31
Gary Marcus
“We still don’t have anything like stable or trustworthy solutions for common sense, reasoning, language, or analogy. Truth is, I have been using this same pie chart for about 5 years, and the situation has scarcely changed.”
Assessment: UnresolvedSaid 2022-05-31
Gary Marcus
“Deep learning is pretty decent at, for example, recognizing objects, but not nearly as good at planning, or reading, or language comprehension.”
Assessment: UnresolvedSaid 2022-05-31
Gary Marcus
“The thing is, it’s easy to convince yourself that AI problems are much easier than they are actually are, because of the long tail problem.”
Assessment: UnresolvedSaid 2022-05-28
Gary Marcus
“As Ernie Davis, Scott Aaronson, and I showed, a week or so after the release, it’s easily fooled, especially with complex sentences and words like “similar” that would require a deep understanding of language to be interpreted correctly”
Assessment: UnresolvedSaid 2022-05-28
Gary Marcus
“Since we know that Imagen can draw images of horses riding astronauts (if the instructions are specific enough) we know that the failure to draw a horse riding an astronaut given the prompt “a horse riding an astronaut” can only have one explanation: the system doesn’t understand what the prompt “a horse riding an astronaut” means.”
Assessment: UnresolvedSaid 2022-05-28
Gary Marcus
“And, no, it wasn’t about common sense at all.”
Assessment: UnresolvedSaid 2022-05-28
Gary Marcus
“(In reality, Imagen has no idea what weight, mass, or density are.)”
Assessment: UnresolvedSaid 2022-05-28
Gary Marcus
“Furthermore, there is no step in the process to try to derive such structures; so it is just wishful thinking to expect them to be fully sensitive to semantics as it relates to syntactic structure.”
Assessment: UnresolvedSaid 2022-05-21
Gary Marcus
“There is no intention to create misinformation—but also no capacity to avoid it, because fundamentally, GPT-3 is a model of how words relate to one another, not a model of how language might relate to the perceived world.”
Assessment: UnresolvedSaid 2022-05-14
Gary Marcus
“Virtually none of the critical problems for AI that cognitive scientists such as Steve Pinker, Judea Pearl, the late Jerry Fodor, and myself have been pointing to for decades is actually solved yet.”
Assessment: UnresolvedSaid 2022-05-14
Gary Marcus
“The bottom line is this, something that AI once cherished but has now forgotten: If we are to build AGI, we are going to need to learn something from humans, how they reason and understand the physical world, and how they represent and acquire language and complex concepts.”
Assessment: UnresolvedSaid 2022-05-14
Gary Marcus
“Systems like DALL-E 2, GPT-3, Flamingo, and Gato appear to be mind-blowing, but nobody who has looked at them carefully would confuse them for human intelligence.”
Assessment: UnresolvedSaid 2022-05-14
Gary Marcus
“Virtually all of the world’s software, from web browsers to spreadsheets to word processors, still depends on knowledge engineering, and Sutton sells that short.”
Assessment: UnresolvedSaid 2022-05-14
Gary Marcus
“So far, in most domains (not all, to be sure) Big Data has triumphed (for now) over careful knowledge engineering.”
Assessment: UnresolvedSaid 2022-05-14
Gary Marcus
“As I have written, I am fine with thinking of Gato as an “Alt Intelligence”— an interesting exploration in alternative ways to build intelligence—but we need to take it in context: it doesn’t work like the brain, it doesn’t learn like a child, it doesn’t understand language, it doesn’t align with human values, and it can’t be trusted with mission-critical tasks.”
Assessment: UnresolvedSaid 2022-05-14
Gary Marcus
“Right now, the predominant strand of work within Alt Intelligence is the idea of scaling.”