Two models of AI oversight - and how things could go deeply wrong
We recorded 18 claims from this post. 3 were checked against evidence. 15 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (18)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Forecast
“What I am worried about is regulatory capture; governments making rules that entrench the incumbents whilst doing too little for humanity.”
Our reading: There is a risk that governments make AI rules that entrench incumbents while doing too little for humanity.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“If big tech writes the rules, without outside input, we are unlikely to wind up with the right rules.”
Our reading: If big tech writes AI rules without outside input, the right rules are unlikely to result.
No horizon
Only if: big tech writes the rules without outside input
Assessment: Can’t be checkedSaid
Conditional forecast
“Putting it bluntly: if we have the right regulation; things could go well. If we have the wrong regulation, things could badly.”
Our reading: With the right AI regulation things could go well; with the wrong regulation things could go badly.
No horizon
Only if: right vs wrong regulation
Assessment: Can’t be checkedSaid
Forecast
“We still have agency here; we can still, I think, build a very positive AI future.”
Our reading: It is still possible to build a very positive AI future.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Press releases and photo opportunities that highlight governments hanging out with the tech moguls they seek to regulate, without independent voices in the room, send entirely the wrong message.”
Our reading: Government photo opportunities with tech moguls without independent voices send the wrong message.
No horizon
Qualifier: without independent voices in the room
Assessment: Can’t be checkedSaid
Causal claim
“But much depends on how much the government stands up to big tech, and a lot of that depends on having independent voices – scientists, ethicists, and representatives of civil society – at the table.”
Our reading: The AI future depends largely on government standing up to big tech, which depends on independent voices being at the table.
No horizon
Assessment: SupportedSaid
Observation
“We also know, for example, that Bing has defamed people, and it has misread articles as saying the opposite of what they actually say, in service of doing so.”
Our reading: Bing has defamed people, and in doing so has misread articles as saying the opposite of what they actually say.
No horizon Evidence as of
Limitation: One connected incident is established; no prevalence or victim count is inferred.
Assessment: UnresolvedSaid
Observation
“I don’t see any reason to think these plugins are in fact safe. (They are far less sandboxed and less rigorously controlled than Apple app store apps.)”
Our reading: Marcus says he does not see any reason to think ChatGPT plugins are in fact safe, noting that they are far less sandboxed and less rigorously controlled than Apple App Store apps.
No horizon Evidence as of
Limitation: The packet lacks a sufficiently complete account of OpenAI's review and enforcement during the target period.
Assessment: SupportedSaid
Observation
“Meanwhile, ChatGPT plugins (produced by OpenAI, which they have a close tie with) open a wide range of security problems: those plugins can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all alarms to any security professional.”
Our reading: ChatGPT plugins open a wide range of security problems, since they can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all of which would alarm any security professional.
No horizon Evidence as of
Limitation: Support concerns the ecosystem's capabilities and attack mechanisms, not every individual plugin.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“This is where the government needs to step up and say “transparency and safety are indeed requirements; you’ve flouted them; we won’t let you do that anymore.””
Our reading: Government should require transparency and safety from AI companies and stop them from flouting those requirements.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We don’t need more photo opportunities, we need regulation – with teeth.”
Our reading: AI needs enforceable regulation rather than photo opportunities.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“More broadly, at an absolute minimum governments need to establish an approval process for any AI that deployed at large scale, showing that the benefits outweigh the risks, and to mandate post-release auditing - by independent outsiders - of any large-scale deployments.”
Our reading: At an absolute minimum, governments need to establish an approval process for any AI deployed at large scale that shows the benefits outweigh the risks, and need to mandate post-release auditing of any large-scale deployment by independent outsiders.
No horizon
Qualifier: at an absolute minimum
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“And governments need to make sure that there are strong liability laws in place, to ensure that if the big tech companies cause harm with their products, they be held responsible.”
Our reading: Governments need to make sure strong liability laws are in place so that, if big tech companies cause harm with their products, they are held responsible.
No horizon
Qualifier: if the big tech companies cause harm with their products
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Goverments should demand that systems only use copyrighted content from content providers that opt-in, and that all machine-generated content be labeled as such.”
Our reading: Governments should require AI systems to use copyrighted content only from opt-in providers and to label all machine-generated content.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“Letting the companies set the rules on their own is unlikely to get us any of these places.”
Our reading: If AI companies set the rules on their own, the needed safeguards are unlikely to result.
No horizon
Only if: companies set the rules on their own
Assessment: Can’t be checkedSaid
Observation
“I actually think that he is at least in part sincere, that it is not just a ploy to keep the incumbents in and small competitors out, that he is genuinely worried about the risks (ranging from misinformation, to serious physical harm to humanity).”
Our reading: Marcus thinks Sam Altman is at least in part sincere in asking for AI regulation, that it is not just a ploy to keep incumbents in and small competitors out, and that he is genuinely worried about AI risks ranging from misinformation to serious physical harm to humanity.
No horizon
Qualifier: I actually think
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“What really matters is what governments around the world come up with by way of regulation.”
Our reading: What really matters is the regulation that governments around the world come up with.
No horizon
Scope: governments around the world
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We would never leave the pharmaceutical industry to entirely self-regulate itself, and we shouldn’t leave AI to do so, either.”
Our reading: The AI industry should not be left to regulate itself, just as pharma is not.
No horizon
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Microsoft reportedly has access to all OpenAI models and can race forward with them regardless of Altman's concerns.
- The two reviews read the claim differently