“Microsoft and OpenAI are producing and deploying products with potentially enormous risks at mass scale, and making extravagant, unreviewed and unreviewable claims about them, without sketching serious solutions to any of the potential problems they themselves have identified.”
Our reading, the sentence that was checked:
Microsoft and OpenAI are producing and deploying products with potentially enormous risks at mass scale, making extravagant, unreviewed and unreviewable claims about them, without sketching serious solutions to any of the potential problems they themselves have identified.
Assessment: Mixed Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, The Sparks of AGI? Or the End of Science?, paragraph 30.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-the-sparks-of-agi-or-the-end-of-science-16908c-a1
We recorded 7 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-03-24. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- potentially (qualifier, stated by the speaker)
From the speaker
- producing and deploying products
- potentially enormous risks
- at mass scale
- extravagant, unreviewed and unreviewable claims
- without sketching serious solutions
- any of the potential problems they themselves have identified
Our interpretation
- The statement describes an ongoing launch-era state. I use March 1–24, 2023, the current reporting period before publication.
- Mass scale means substantial public deployment, rather than a laboratory-only experiment. No numerical threshold was supplied.
- Sketching serious solutions means describing substantive measures directed at acknowledged problems. The measures need not eliminate all risks or prove that deployment is safe.
- Unreviewed is distinguished from unreviewable: publication as a preprint does not establish independent peer review, while behavioral testing does not establish reproducibility of the original model or training process.
- Extravagant and enormous are qualitative judgments. I assess documented claims, risks, access, and proposed responses without inventing numerical thresholds for those adjectives.
- Potentially qualifies the risks; it is not a conditional forecast.
Counts as supported if: Contemporaneous records show large public deployment and serious acknowledged risks, claims lacking meaningful independent scrutiny, and no substantive proposed response to any acknowledged problem.
Counts as contradicted if: Concrete published mitigation methods or implemented controls contradict the no-solutions component. Demonstrated independent scrutiny would contradict absolute unreviewability, but not necessarily establish peer review or full reproducibility.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Mixed
Part of the claim holds and part does not, and the record says which part is which.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Large public deployment and acknowledged safety risks are supported. Microsoft reported millions of preview users and substantial chat activity, while OpenAI documented harmful-output risks. But the assertion that neither company had even sketched serious responses is contradicted: OpenAI described technical safety measures, and Microsoft reported implemented conversation and behavior controls. These measures did not amount to complete solutions. The broader judgments about extravagance and unreviewability are not fully settled by these records.
Limitations
- The result treats substantive mitigations as satisfying sketching serious solutions.
- The risk magnitude and effectiveness of the controls are not independently established.
- The review-status component remains only partially assessed; preprint status alone does not prove absence of review.
- Deployment evidence establishes large-scale Bing AI use, not an exact GPT-4 user count across both companies.
Evidence (5 items)
Supports the claim: Microsoft Bing,
45 million total chats
Why it matters here: Microsoft reported millions of active preview users and more than 45 million cumulative chats. This supports substantial public deployment; the separate total for all Bing users is not treated as AI-chat usage.
- Publisher
- Microsoft Bing
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: OpenAI,
compromised or vulnerable
Why it matters here: The launch documentation acknowledges insecure code, harmful advice, bias, and other safety risks. This supports the presence of potentially serious harms, without establishing their aggregate magnitude or that they occurred.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Goes against the claim: OpenAI,
controllable and useful.
Why it matters here: The report describes reinforcement learning, safety-specific training, expert testing, and abuse monitoring as responses to acknowledged problems. Their publication contradicts the absence of substantive proposals, while the report also acknowledges remaining failures.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Goes against the claim: Microsoft Bing,
your feedback is helping us improve.
Why it matters here: Microsoft described an implemented six-turn conversation limit and changes addressing defensive or adversarial responses. These were concrete controls, although this disclosure does not independently demonstrate their effectiveness.
- Publisher
- Microsoft Bing
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Microsoft Research authors via arXiv,
rising capabilities and implications
Why it matters here: The paper makes a broad claim that an early GPT-4 could be considered an incomplete AGI system. Its preprint record establishes the claim and publication timing, but cannot establish the absence of every form of review or prove unreviewability.
- Publisher
- Microsoft Research authors via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Dated review records and independent replication attempts for the specific scientific claims would settle more of the review-status component. Independent evaluations of the published controls would clarify their effectiveness.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, The Sparks of AGI? Or the End of Science?
Microsoft and OpenAI are producing and deploying products with potentially enormous risks at mass scale, and making extravagant, unreviewed and unreviewable claims about them, without sketching serious solutions to any of the potential problems they themselves have identified.
- Where
- paragraph 30
- Published
- 2023-03-24
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-the-sparks-of-agi-or-the-end-of-science-16908c-a1: Current. Outcome: Mixed. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-the-sparks-of-agi-or-the-end-of-science-16908c-a1.