Scientists, please don’t let your chatbots grow up to be co-authors
We recorded 11 claims from this post. 3 were checked against evidence. 8 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (11)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The worst thing about ChatGPT’s close-but-no-cigar answer is not that it’s wrong. It’s that it seems so convincing.”
Our reading: The worst thing about ChatGPT's wrong answer in the cited tweet is not that it is wrong but that it seems so convincing.
No horizon
Scope: ChatGPT's close-but-no-cigar answer in the cited tweet
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I sincerely hope this trend won’t last.”
Our reading: The trend of listing ChatGPT as a co-author should not continue.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“You wouldn’t make your spell-checker or your spreadsheet a co-author.”
Our reading: ChatGPT should not be made a co-author, just as you wouldn't make your spell-checker or spreadsheet a co-author.
No horizon
Assessment: Can’t be checkedSaid
Observation
“ChatGPT is a tool, but not a person. It’s at best, more like a spell-checker, or a grammar-checker, or a stats package, than it is like a scientist.”
Our reading: ChatGPT is a tool, closer to a spell-checker or stats package than to a scientist.
No horizon
Assessment: UnresolvedSaid
Observation
“It is not contributing genuine ideas, nor coming up with carefully controlled experimental designs, nor summing up prior literature in thoughtful ways.”
Our reading: ChatGPT does not contribute genuine ideas, design controlled experiments, or thoughtfully summarize literature.
No horizon Evidence as of
Limitation: Proposal ratings do not establish scientific validity, accurate literature synthesis, or sound experimental controls [e1].
Assessment: UnresolvedSaid
Observation
“ChatGPT simply cannot reason well enough to to do this; as I have documented in previous essays, it has demonstrably misunderstood physics, biology, psychology, medicine, and so forth.”
Our reading: ChatGPT cannot reason well enough to make a substantive scientific contribution and has misunderstood physics, biology, psychology, and medicine.
No horizon Evidence as of
Limitation: The evidence does not independently establish misunderstandings in physics and psychology [e1, e2].
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Given ChatGPT’s current understanding level of scientific understanding, assigning authorship to it is disrespectful to the scientific process itself.”
Our reading: Given its current scientific understanding, assigning authorship to ChatGPT disrespects the scientific process.
No horizon
Qualifier: given current level of scientific understanding
Assessment: Can’t be checkedSaid
Rhetoric
“If you declare it as a co-author, it says you are more interested in being on trend, than in being on target.”
Our reading: Declaring ChatGPT a co-author says that you are more interested in being on trend than in being on target.
No horizon
Qualifier: if you declare it as a co-author
Assessment: SupportedSaid
Observation
“ChatGPT has proven itself to be both unreliable and untruthful. It makes boneheaded arithmetical errors, invents fake biographical details, bungles word problems, defines non-existent scientific phenomenon, stumble over arithmetic conversion, and on and on.”
Our reading: ChatGPT is unreliable and untruthful: it makes arithmetic errors, fabricates biographical details, bungles word problems, and invents scientific phenomena.
No horizon Evidence as of
Limitation: Selected demonstrations do not establish an overall error rate or intentional deception [e1, e2, e3, e4].
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If you treat ChatGPT as a sentient being, when it clearly is not, you are misleading the public into thinking ChatGPT is sentient; rather than communicating good science, you are hyping bullshit. Just stop.”
Our reading: ChatGPT is clearly not sentient, and scientists should stop treating it as such because doing so misleads the public.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Friends don’t let friends co-author with ChatGPT.”
Our reading: Scientists should not co-author papers with ChatGPT.
No horizon
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.