“Reid spent about 45 minutes looking into this earlier today (why not more? see below) with images generated by Midjourney, and it quickly became apparent that using ChatGPT as intern to trace down sources is, well, pretty intermittent.”
Our reading, the sentence that was checked:
In a 45-minute test, multimodal ChatGPT was unreliable at identifying the copyrighted sources of Midjourney-generated images.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, No, multimodal ChatGPT is not going to “trivially” solve Generative AI's copyright problems, paragraph 23.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-no-multimodal-chatgpt-is-not-going-5a9763-a2
We recorded 9 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2024-01-24. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- tested on Midjourney-generated images (qualifier, stated by the speaker)
From the speaker
- Reid spent about 45 minutes
- earlier today
- images generated by Midjourney
- using ChatGPT as intern to trace down sources is ... pretty intermittent
Our interpretation
- Plain reading: in an informal test of about 45 minutes on 2024-01-24, ChatGPT correctly identified the sources of some Midjourney images but missed or gave vague answers on others.
- Settling this needs independent records of the test or a faithful reproduction. The speaker's own screenshots cannot count as evidence that he was right.
Counts as supported if: Independent records or a reproduction from that time show ChatGPT getting some Midjourney image sources right and failing on others.
Counts as contradicted if: Independent records or a reproduction show ChatGPT consistently identifying the sources of the images tested.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The claim concerns a test of about 45 minutes that Marcus's post says Reid Southen ran with Midjourney images. The only record of that test is the speaker's own post, and that cannot count as evidence that he was right. The evidence packet has no independent record of the test and no reproduction using the same images. AI Snake Oil's post, available 2024-01-23 (e1), reports that ChatGPT-4 recognized all nine images from an earlier Marcus and Southen article. That fits the statement that ChatGPT 'nailed some images'. But those were different images, so e1 says nothing about the harder cases in the claimed test. The evidence neither confirms nor contradicts the claim that ChatGPT's performance in that test was intermittent.
Limitations
- The packet has no independent record of the images Southen tested or of ChatGPT's answers to them.
- The AI Snake Oil test (e1) used a different set of images, so its results do not carry over to the claimed test.
- 'Pretty intermittent' has no threshold, so it is unclear how many failures would count.
Evidence (1 item)
Context only: AI Snake Oil (Narayanan and Kapoor),
ChatGPT-4, which is multimodal, straightforwardly recognizes all of them
Why it matters here: An independent test found ChatGPT-4 recognized all nine iconic images. That fits Marcus's statement that ChatGPT 'nailed some images'. It says nothing about the harder images Southen tested.
- Publisher
- AI Snake Oil (Narayanan and Kapoor)
- Kind of source
- Secondary summary
- Describes events of
- Public from
What would settle it
Independent records of the Midjourney images used in the 2024-01-24 test and ChatGPT's answers, or a third-party reproduction with the same images on the ChatGPT-4 model available at that time.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, No, multimodal ChatGPT is not going to “trivially” solve Generative AI's copyright problems
Reid spent about 45 minutes looking into this earlier today (why not more? see below) with images generated by Midjourney, and it quickly became apparent that using ChatGPT as intern to trace down sources is, well, pretty intermittent.
- Where
- paragraph 23
- Published
- 2024-01-24
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-no-multimodal-chatgpt-is-not-going-5a9763-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-no-multimodal-chatgpt-is-not-going-5a9763-a2.