“Everybody spending millions of dollars on AI tokens is experimenting. As I’ve discussed previously, nobody really knows how to measure the ROI of AI, and the naturally-chaotic nature of LLMs makes it impossible to measure how much it might even cost:”
Our reading:
Everybody spending millions of dollars on AI tokens is experimenting; nobody really knows how to measure AI's ROI, and LLMs' chaotic nature makes it impossible to measure how much it might cost.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The claim says nobody spending millions on AI tokens can measure the return, and that the cost cannot be predicted. Uber fits it. Its COO said he cannot draw a line from rising Claude Code use to useful features, and Uber used up its 2026 AI coding budget in four months. But one firm cannot prove an 'everybody' or 'nobody' claim. The likeliest counterexample is weak. 8x8 told WIRED in June 2026 that Claude helped it save about $5M a year, more than its Claude bill. That bill may be under the 'millions' threshold, the savings are self-reported, and the report came after the post. I found no clear case before the post of a company spending millions on tokens and publishing a measured return on it, and no evidence that settles the claim either way.
Caveats
- Sweeping universal wording; only a clear counterexample could settle it.
- Search coverage of company ROI disclosures is incomplete.
Evidence (3 items)
Supports the claim: Fortune,
“If you’re not actually able to draw a direct line to how [many] useful features and functionality you’re shipping to your users, that trade becomes harder to justify,” Macdonald said.
Why it matters: A large token spender's executive says he cannot link the spending to output. This fits the claim but is one company.
- Where
- paragraph 5
- Publisher
- Fortune
- Kind of source
- News report
- About events on
- Published
Supports the claim: Fortune,
the firm had already burnt through its entire 2026 AI coding tools budget in just four months
Why it matters: Uber did not forecast its token costs well. This fits the 'hard to predict cost' reading for one firm.
- Where
- paragraph 4
- Publisher
- Fortune
- Kind of source
- News report
- About events on
- Published
Goes against the claim: AITopics (summary of WIRED),
the company estimates it has saved about $5 million in annual costs by canceling subscriptions to dozens of software and educational tools
Why it matters: A company claims a measured saving above its Claude bill. But the bill is only 'well below' $5M, so it may not be a 'millions on tokens' spender. The report is also after the post and self-reported, so it is not decisive.
- Where
- summary paragraph
- Publisher
- AITopics (summary of WIRED)
- Kind of source
- News report
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- spending millions of dollars on AI tokens (their words)
Where we had to interpret
- Plain reading: as of the post, no company spending millions on AI tokens can credibly show a measured return on that spending, and none can reliably predict what its token use will cost.
- 'Experimenting' is read loosely: spending without an established, measured payoff. It is not read as a formal pilot status.
- 'How much it might even cost' is read as predicting or budgeting future cost, not metering past spend, since the passage that follows is about budgeting.
- He named no period. I use the state around the post: April 1 to May 26, 2026.
Right if: Reporting shows large token spenders consistently unable to measure returns or to predict costs, with no credible counterexample.
Wrong if: A company spending millions of dollars a year on AI tokens publicly reports a specific, measured financial return on that spending (savings or profit above its token bill), or shows it can reliably forecast its token costs.
We wrote this test after the outcome was known.
What would settle it
A company spending millions a year on AI tokens publicly reporting, before or around May 2026, a specific financial return that exceeds its token spend, or systematic evidence that large spenders cannot measure returns.
We found 42 claims in this post. See them all.
Full record
Source and dates
Where's Your Ed At, Revenge of The Business Idiot, paragraph 97.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2026-05-26.
Where it was said (1)
Said once in the posts we read.
: Where's Your Ed At, Revenge of The Business Idiot
Everybody spending millions of dollars on AI tokens is experimenting. As I’ve discussed previously, nobody really knows how to measure the ROI of AI, and the naturally-chaotic nature of LLMs makes it impossible to measure how much it might even cost:
- Where
- paragraph 97
- Published
- 2026-05-26
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.