“It is not contributing genuine ideas, nor coming up with carefully controlled experimental designs, nor summing up prior literature in thoughtful ways.”
Our reading, the sentence that was checked:
ChatGPT does not contribute genuine ideas, design controlled experiments, or thoughtfully summarize literature.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Scientists, please don’t let your chatbots grow up to be co-authors, paragraph 16.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-scientists-please-dont-let-your-chatbots-bde4b4-a2
We recorded 11 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-01-14. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- not contributing genuine ideas
- nor coming up with carefully controlled experimental designs
- nor summing up prior literature in thoughtful ways
- ChatGPT
Our interpretation
- Assess the early ChatGPT available during the six-week history described in the post, through January 14, 2023.
- The ordinary reading concerns substantive intellectual assistance with scientific work, including work done under human guidance. It does not require autonomous discovery or eligibility for authorship.
- Genuine ideas means scientifically meaningful research suggestions; thoughtful summaries require accurate synthesis; controlled designs require appropriate controls and treatment of confounding factors.
- Later publications can establish earlier performance only when their relevant outputs can be dated to the target period.
Counts as supported if: Dated outputs and substantive expert evaluations establish the claimed deficiencies across idea generation, controlled experimental design, and literature synthesis. Isolated failures alone do not establish categorical inability.
Counts as contradicted if: A documented contemporary example supplies a scientifically meaningful idea, a sound controlled experimental design, or an accurate substantive literature synthesis, with independent assessment establishing that contribution.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The finance study reported favorable ratings for generated research proposals, with the strongest ratings for ideas and weaker results for literature reviews and testing frameworks [e1]. Those ratings do not establish categorical deficiencies across all three activities [e1]. The accessible revised study also lacks precise output dates and sufficient validation to establish a decisive counterexample from November 30, 2022–January 14, 2023 [e1].
Limitations
- Proposal ratings do not establish scientific validity, accurate literature synthesis, or sound experimental controls [e1].
- The accessible study does not identify which relevant outputs preceded January 14, 2023 [e1].
Evidence (1 item)
Context only: Finance Research Letters; copy hosted by Abu Dhabi University,
‘succeed’.
Why it matters here: Reviewers rated generated research proposals favorably, particularly their ideas; literature synthesis and testing frameworks were weaker. This challenges the blanket dismissal, but precise output dates and validation of the proposed scientific contributions remain insufficient for a decisive historical counterexample.
- Publisher
- Finance Research Letters; copy hosted by Abu Dhabi University
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
What would settle it
Dated prompts and outputs from November 30, 2022–January 14, 2023, with independent expert assessment and a clear record of human contributions. A verified meaningful idea, sound controlled design, or accurate substantive literature synthesis would contradict the claim; support would require broad evidence establishing the asserted deficiencies across all three activities.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Scientists, please don’t let your chatbots grow up to be co-authors
It is not contributing genuine ideas, nor coming up with carefully controlled experimental designs, nor summing up prior literature in thoughtful ways.
- Where
- paragraph 16
- Published
- 2023-01-14
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-scientists-please-dont-let-your-chatbots-bde4b4-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-scientists-please-dont-let-your-chatbots-bde4b4-a2.