“To be sure, large language models are, in a lots of ways, an incredible advance over the AI of the 1970s. They can engage in a much broader range of tasks, and generally they need less hand-tinkering (Though they still need some – that’s what reinforcement learning with human feedback is really about.)”
Our reading, the sentence that was checked:
In a lot of ways, large language models are an incredible advance over the AI of the 1970s: they can engage in a much broader range of tasks and generally need less hand-tinkering, though they still need some, which is what reinforcement learning with human feedback is really about.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Hello, Multimodal Hallucinations, paragraph 23.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-hello-multimodal-hallucinations-85ff6e-a1
We recorded 11 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-10-21. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- in a lot of ways (qualifier, stated by the speaker)
- generally (qualifier, stated by the speaker)
From the speaker
- an incredible advance over the AI of the 1970s
- a much broader range of tasks
- generally they need less hand-tinkering
- Though they still need some
- reinforcement learning with human feedback
Our interpretation
- Assess the two concrete comparisons: task breadth and reliance on manual customization. Treat 'incredible advance' as an appraisal of those comparisons, not a separate measurable claim.
- The plain reading of 'less hand-tinkering' is less task-specific programming and customization to handle different tasks, not fewer total development hours or less computing.
- Use representative general-purpose LLMs and the specialized 1970s systems identified in the context, particularly SHRDLU. The claim does not assert superiority on every task.
- Interpret the RLHF parenthesis as acknowledging continued human intervention in shaping model behavior, not as a complete technical definition of RLHF.
- For the ongoing state, use the current calendar year through publication, 2023-01-01 to 2023-10-21. Technical capability reports have no regular quarterly reporting period. Treat the 1970s documentation as historical comparison context.
Counts as supported if: Contemporary technical reports demonstrate that reusable LLMs perform across substantially more task domains than the specialized historical systems, without separately programming each task, while documenting continued human-guided adjustment such as RLHF.
Counts as contradicted if: Comparable documentation shows that contemporary LLMs remain similarly confined in task breadth or generally require separately hand-built task procedures comparable to the historical systems. Evidence that RLHF does not involve human-guided adjustment would contradict the parenthetical explanation.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence supports the ordinary task-specific reading. SHRDLU operated with a supplied model of a restricted setting. Contemporary Google research documents models handling new tasks through instructions, and GPT-4 was evaluated across varied exams without exam-specific training. These records support broader task coverage and less need to program each task separately. OpenAI also confirms continued human-feedback training, matching Marcus's qualification. The comparison does not establish lower total development labor or superiority on every task.
Limitations
- The conclusion depends on reading 'hand-tinkering' as task-specific customization.
- Contemporary capability evidence comes from model developers.
- Prompt design, instruction-data preparation, and human feedback remain substantial work.
- The historical baseline is representative rather than exhaustive.
Evidence (4 items)
Supports the claim: Google Research,
Language models are now capable of performing many new natural language processing (NLP) tasks by reading instructions, often that they hadn’t seen before.
Why it matters here: The report describes instruction-based generalization and evaluations across diverse held-out tasks. Compared with the bounded historical setting, this supports broader reuse with less task-specific programming, while its training-method discussion still documents substantial human design work.
- Where
- Opening paragraph
- Publisher
- Google Research
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: OpenAI via arXiv,
We did no specific training for these exams.
Why it matters here: The report evaluates the same model across professional and academic exams without exam-specific training, supporting breadth and reduced task-specific customization. It also describes format-specific prompts and possible training-data overlap, so this is not evidence of zero manual work.
- Publisher
- OpenAI via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: OpenAI,
To align it with the user’s intent within guardrails, we fine-tune the model’s behavior using reinforcement learning with human feedback (RLHF).
Why it matters here: The developer explicitly documents human-feedback training to shape behavior. This supports Marcus's qualification that manual intervention remains, without equating RLHF with hand-written task rules.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: MIT Artificial Intelligence Laboratory,
The program is given a detailed model of the knowledge needed by a simple robot having only a hand and an eye.
Why it matters here: Winograd describes a supplied model for a restricted robot setting. The abstract documents dialogue, deduction, and manipulation of toy objects, establishing a capable but bounded historical baseline.
- Publisher
- MIT Artificial Intelligence Laboratory
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Hello, Multimodal Hallucinations
To be sure, large language models are, in a lots of ways, an incredible advance over the AI of the 1970s. They can engage in a much broader range of tasks, and generally they need less hand-tinkering (Though they still need some – that’s what reinforcement learning with human feedback is really about.)
- Where
- paragraph 23
- Published
- 2023-10-21
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-hello-multimodal-hallucinations-85ff6e-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-hello-multimodal-hallucinations-85ff6e-a1.