Method

What this register is

A curated, public record of AI claims made by Gary Marcus and Ed Zitron. For each selected claim it shows what was said, what the statement commits the speaker to, what evidence arrived, what assessment that evidence supports, and what changed afterwards.

It is not a rating of either person. There is no accuracy percentage, ranking, weighted score, or verdict on anyone’s intent. Counts describe the selected register, filtered as you choose. They never describe a person’s whole output. Leaving a statement out says nothing about whether it was right, wrong, or important.

Other people can appear as sources, as quoted third parties, or as the subject of a claim. They never become a third subject. Adding a subject would need a new policy version.

Where statements come from

Statements come from the public long-form writing of the people covered: Marcus on AI, Where's Your Ed At. The coverage page lists the date windows and reports, for each one, how many posts were discovered, fetched, failed, read in full, and selected.

Paywalled posts are counted and not read. A paywall preview is never treated as the full text. Each quotation shows the date this site observed the source. Nothing here claims that today’s wording was captured years ago unless an archived copy from then exists.

Every claim shows the speaker’s own words, quoted in full (at most 80 words per quotation), with a link to the original. They are quoted for commentary and criticism and remain the author’s.

How claims are selected

Selection is fixed before any outcome is graded. Each selected candidate records the reason it was admitted, in this order of priority:

1. Matured dated forecast: an explicit date or relative deadline that has passed.
2. Open dated forecast: an explicit deadline that has not passed.
3. Central recurring proposition: repeated in three or more posts, and counted once.
4. Evidence changed: new primary evidence arrived inside the window.
5. Declared lead: named in the project’s source inventory.

The eligible inventory for each person is frozen, with its hash, before grading. A candidate belongs to the first reason above that it qualifies for. Candidates are taken reason by reason in that order. Within one reason, the order is the sha256 hash of the policy version plus the candidate id. Taking stops at 20 candidates per person, or when the inventory runs out. The selected ids and what happened to each one are published, so anyone can reproduce the selection.

A candidate is never chosen because its outcome is known to embarrass or flatter the speaker. It is never dropped after grading because of its outcome.

Statements, claims, and assessments

A statement is what someone said at a particular time in a particular source. A claim is the proposition that statement commits them to. An assessment is this site’s judgment of the claim at a stated evidence cutoff. They are separate records, and each claim page shows all three.

A claim rests only on statements the person endorsed. Quoting a critic, reporting what someone else said, or a reader’s comment is not the person’s claim.

A compound statement can produce several claims. A recommendation and the factual premise behind it are split. Topics help navigation only and never carry a verdict.

The same proposition under the same conditions is one claim, however often it was said. Each occurrence is listed on the claim page, and the claim counts once. A materially changed commitment (a new deadline, population, metric, condition, quantifier, or target) is a new claim linked to the old one, and the old one keeps its own deadline. When a speaker changes their wording, the record notes it as the speaker’s revision, without any accusation of moving goalposts. Revisions under one claim id are reserved for this site’s own corrections.

Kinds of claim

• Observation: a statement about how things are or were.
• Forecast: a statement about what will happen.
• Conditional forecast: a forecast that depends on something else happening first. That condition (the antecedent) is kept.
• Causal claim: a statement that one thing caused or causes another.
• Recommendation or value judgment: a recommendation or a value judgment. A date inside a recommendation does not make it a forecast.
• Rhetoric: an expression that commits to no testable proposition.

Criteria are fixed before grading

Before any outcome is graded, the claim’s wording and its resolution criteria are frozen. The criteria separate what the speaker said from what this site added to make the claim testable. The added part is labelled “our interpretation” on every claim page.

Only dates and thresholds the speaker stated count as commitments. A missing deadline stays missing, and a vague phrase is shown as written. When our interpretation would decide the result, the claim is not scored.

Criteria written after the outcome was public are labelled retrospective. A record is prospective only when it is a forecast whose criteria were frozen before its deadline, while its outcome was not yet public, and every decisive piece of evidence became public after the freeze. A deadline that has not yet passed does not by itself make a record prospective. Any later change to frozen criteria creates a new version and needs a new review.

What an assessment says

Each assessment answers one question:

• Was it true for the period described? Evidence must be dated precisely enough to fall inside the period the claim describes. Evidence about a later period is context, never support or contradiction.
• Did the forecast come true? Judged against the forecast’s original conditions and deadline.
• Does the evidence support the cause? Needs evidence suited to showing a cause. The limits are stated.
• Not graded on evidence Recommendations, value judgments, rhetoric, hedged forecasts, and claims too open to test. They are recorded and attributed, not scored.

It gives one of five outcomes:

• Supported: The dated evidence matches what the claim committed to, under the criteria frozen before grading.
• Contradicted: The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
• Mixed: Part of the claim holds and part does not, and the record says which part is which.
• Unresolved: The record was checked, and the evidence available by the cutoff does not settle it.
• Can’t be checked: The statement is a recommendation, a value judgment, or too open to test. It is recorded, not scored.

The state of the deadline is shown separately from the outcome. “Due” and “deadline passed” are computed for the release’s reference date (2026-10-29), never for the day you read the page.

• No horizon: The question has no deadline.
• No deadline stated: The speaker gave no deadline. None is added.
• Not yet due: The deadline has not passed. The event not having happened yet is not a failure.
• Deadline passed: The deadline has passed. This queues a review; it never sets an outcome by itself.
• Settled before its deadline: The forecast was settled before its deadline by an event its criteria allow. The record names that event and its date.

Rules that decide outcomes

• A deadline passing never sets an outcome. It queues a review. When the review completes, the “review due” notice clears, even if the outcome stays unresolved.
• A conditional forecast is scored only when evidence shows that its condition happened. If the condition did not happen, or nobody can tell, the forecast is neither confirmed nor contradicted.
• Hedged forecasts are recorded and shown next to what happened, but not scored: one outcome cannot grade a probability. That covers probability and possibility words (“may”, “might”, “could”, “likely”, “probably”, “a good chance”) and forecasts the author filed under medium or low confidence. Unqualified expectation or high confidence about a definite outcome (“I expect”, “I predict”, “I am confident that”, “will”, “won’t”) is a forecast and is scored.
• Only outcome evidence decides an outcome. The speaker’s own statement proves what was said, never that it was true.
• A forecast settled before its deadline names the event that settled it, the event’s date, and how it meets the criteria.
• Financial evidence counts only when it measures exactly the same quantity over exactly the same period: the same metric, currency, units, accounting basis, and dates. A claim with no stated accounting basis is not scored. Revenue is not annualised run rate. Financing is not revenue. Operating income is not net income. A cash balance is not cash flow. Multi-year commitments are not all payable now. A better 2026 quarter does not contradict an accurate statement about 2025. Any arithmetic shows its inputs, formula, and rounding.
• Hindsight is excluded: the evidence for an assessment includes only what was public by its cutoff.

How AI models review a record

The method follows the automated approach of Because of AI. One AI model drafts the record. Two models from different companies, Anthropic and OpenAI, then review the same frozen claim, criteria, and primary-source evidence. They work independently: neither sees the other’s reasoning or a requested answer. A model from a different company than the drafter then checks the final text that readers see against the frozen inputs. The reviewing models run with no tools: no shell, no files, no network. Reviews for the initial build ran through each provider’s own subscription client with every tool switched off, checked before each run by a test that plants a decoy file and a hostile instruction; ongoing reviews use the providers’ APIs. Every review record says which route produced it.

Code, not a model, decides whether a record publishes. Before a record publishes, code requires all of these:

• one of the two subjects, a valid record, correct attribution, and a quotation found exactly in the observed source;
• enough of the original context, and evidence that was available by the cutoff;
• criteria that keep the speaker’s conditions, deadline, metric, and scope;
• valid, independent reviews from both companies, with no unresolved disagreement and no malformed answer;
• a check of the final public text by a company other than the drafter’s;
• an exact match between the published text and what was checked. Any edit after the check cancels it.

Agreement between models is a check on the record, not proof that a claim is true or false. This is automated review. No person is required to approve each record before it is published. Models can be wrong, and two models can make the same mistake. Code can check that a quotation matches its source and that nothing changed after review. It cannot check judgment.

The models are made by companies that some claims on this site are about. That conflict is stated on every claim page, and stated specifically beside any claim about Anthropic or OpenAI. A company’s statements about itself are labelled as company disclosures, not independent findings. Financial arithmetic is shown so that you can redo it.

Claims that can’t be checked

Before any grading, a model reads each post and lists the claims in it. Then a model from each provider, Anthropic and OpenAI, screens every claim on its own. Each screen answers four questions: did the author make this claim and stand behind it, does our sentence say what the author said, is the kind of claim right, and could public evidence settle it at all.

When both screens agree on the first three and both find that evidence could not settle the claim, it is published as “Can’t be checked” with the reasons they gave. Each reason is one of these:

• A recommendation or a value judgment.
• Rhetoric that commits to nothing testable.
• Stated as a chance or a possibility, or at medium or low confidence.
• Too vague to test: informed readers would test it in different ways and reach different answers.
• About something no public evidence could show, such as a private intention or a counterfactual.

A claim screened out this way is not checked against evidence, and no model writes its text. Code composes the text from the two screens, so there is no separate check of the final text. Code still checks that the quotation matches the source and that the person made the statement, as it does for every record. If either screen finds the claim could be checked, or the screens disagree about anything, the claim is graded in full as described above.

How the homepage answer is worked out

The answer for each person counts only their claims the evidence settled: supported, mixed, or contradicted. A mixed claim counts as half supported. With fewer than 10 settled claims the answer is “Too early to say.” Otherwise the share decides the words: “Mostly right” at 80% or more, “More right than wrong” at 60% or more, “Right about as often as wrong” at 40% or more, “More wrong than right” at 20% or more, and “Mostly wrong” below that.

Claims that are unresolved, that can’t be checked, or that are held are left out, and the homepage says how many. A claim that can’t be checked is not counted as wrong. The topic and year breakdowns use the same counts and show a group only once 10 of its claims have settled.

What is held, and why

A record publishes only when every check passes. Otherwise it is held with one of these reasons, and the held list is public:

• The source could not be read.
• Only part of the source could be read.
• Attribution is uncertain.
• The quotation did not match the source.
• The criteria are ambiguous.
• The evidence is incomplete.
• The two reviews read the claim differently.
• The two reviews reached different assessments.
• A required check could not run.
• A review returned an unusable answer.
• The text changed after it was checked.
• The run stopped at its budget.
• A dispute is open.
• The record does not meet the current rules.

Held is a publication state, not a finding. A held record is not a contradicted claim. A provider outage never becomes an “unresolved” verdict. A well-checked case with genuinely insufficient evidence can publish as unresolved, but an unchecked case cannot publish at all.

There are no grandfathered records. Every public assessment passed the checks under the policy version it names.

Corrections, disputes, and history

Every change is an event with a date and a reason. Events by the speaker (a clarification, a revision, a retraction) and events by this site (a correction, a reassessment, a withdrawal, an under-review flag) never share a label. A formatting edit is not an event.

You can dispute a record through a GitHub issue. Each claim page has a link that fills in the claim and the assessment version. GitHub requires a free account, and issues are public. The editor triages disputes. A dispute enters a queue: it cannot change the site, run code, or reach credentials. Each one is upheld, rejected, or kept under review, with a reason. A dispute is never shown as a finding.

A verified problem with a published assessment can flag it as under review. A quotation that does not match, a failed attribution, a source-integrity failure, or an extraction error can withdraw it. Neither step adds a new verdict. Restoring a withdrawn assessment needs a new assessment through every check. Old versions stay readable at their own addresses, with a clear notice.

Privacy and rights

Privacy, rights, or safety problems with quoted text go to a private contact route, not a public issue. Write to [email protected].

The editor can remove the affected text from every current and historical page, with a dated, logged notice and no new verdict. Copies already distributed cannot be recalled.

Policy versions

The current policy is 2026-10-28.7. Every assessment names the policy version it passed under and is never described as passing a newer one. When a policy change could alter an existing outcome, that assessment is flagged for review. Reaffirming it runs every check under the new version.

• 2026-10-28.7 (2026-10-28), current: Full corpus. Every extracted claim is first screened by both providers, a post at a time, for attribution, faithfulness, kind, and whether public evidence could settle it at all. A claim both find uncheckable is published as not_adjudicable with the stated reasons, without an evidence search; its public text is composed by code from the screen, so no final-text check is needed. Every other claim is graded on its own: evidence, two full reviews, and a final-text check. Hedged claims are recorded and not scored; the page no longer promises to show what happened for each one.
• 2026-10-28.6 (2026-10-28): Owner decision: a firm claim that names no measure (for example 'the bubble will begin to burst within 12 months') is scored on the plain reading an informed reader would take. The reading is stated as our interpretation and the page says the record was scored on a plain reading. Hedged and probabilistic language is still not scored.
• 2026-10-28.5 (2026-10-28): Owner decision: every claim shows the speaker's full words (excerpts up to 80 words) and the per-source quoting cap is off. Also: when both reviews and the draft find a claim cannot be scored, either allowed axis label is accepted.
• 2026-10-28.4 (2026-10-28): Graded confidence is not a commitment. A forecast that its author files under "medium confidence" or "low confidence" is not scored, like "likely". Prompted by a pilot draft from Marcus's 2025 predictions, which he grouped by confidence level. Decided before any record was published.
• 2026-10-28.3 (2026-10-28): Defines which confidence language is scored. Prompted by the first pilot review, where the two reviewers split on "I am more confident than ever that GPT-5 won't land this year". Expressed confidence or expectation about a definite outcome is a forecast and is scored; probability and possibility words are not. Decided before any record was published.
• 2026-10-28.2 (2026-10-28): Fixes from the cross-vendor audit before any assessment existed. Conditionals score only on a demonstrated antecedent; hedged forecasts are not scored; only outcome evidence decides; financial evidence must match currency, units, basis, and the exact period; observation evidence must fall inside the period; early resolution names its event; prospective needs a pre-outcome freeze; selection order and stopping rule fixed; per-source quoting cap.
• 2026-10-28.1 (2026-10-28): First policy. Two subjects, curated register, no aggregate score, two-provider review, no grandfathered verdicts.

Assessments in this release, by the policy they passed under: 2026-10-28.7 (5055), 2026-10-28.5 (20).

Where this started

The project grew out of the essay The most expensive kind of correct and the dataset of 2,218 claims extracted from Gary Marcus’s writing (snapshot of 2026-03-02), which this register uses only as a list of candidates.

No correction to the original analysis has been recorded yet. Corrections will be listed in this section, and none of them adds a third subject.