Expert Excluded After ChatGPT Hallucinates Citations in Minnesota Deepfake Case

Readers arriving through the phrase “melissa hortman husband murder sentence minnesota” are likely looking for a different federal-court story. The Hortman/Boelter murder sentencing is separate from Kohls v. Ellison. Both sit in the Minnesota federal-court risk environment, but this article is about the Jan. 10, 2025 order in Kohls: an AI-deepfake case in which an expert declaration was excluded after ChatGPT-4o generated fake scholarly citations.

The order’s importance is not that a lawyer, expert, or court encountered a generative-AI error. That story is now familiar. The harder point is procedural: once fabricated authorities entered an expert declaration signed under penalty of perjury, the problem moved from “AI hallucination” to expert reliability, attorney supervision, and the court’s ability to trust the record.[1]

What The Court Actually Did

Kohls challenged Minnesota’s law addressing election-related deepfakes. Minnesota submitted a declaration from Stanford professor Jeff Hancock, an expert on misinformation and technology, to support the state’s position in the First Amendment dispute. The declaration discussed AI-generated misinformation and cited academic work in support of its analysis.[1][2]

The opposing side found the defect that should make any trial team stop reading and start verifying: the declaration contained two non-existent academic articles and one misattribution. The court found that Hancock had used ChatGPT-4o to assist in preparing the declaration and that the cited sources at issue were AI-generated hallucinations.[1]

Judge Provinzino excluded the entire Hancock declaration. She did not merely strike the bad citations, accept a corrected version, or treat the episode as an embarrassing footnote. The court held that citing fake, AI-generated sources “shatters his credibility with this Court,” and denied Minnesota’s request for leave to submit an amended declaration.[1]

That is the operational center of the ruling. The sanction landed on the evidentiary use of the expert’s declaration. The court’s reasoning then reached outward, toward counsel’s obligations when declarations are prepared with the assistance of generative AI.

The Defect Was Not A Bad Footnote

The fabricated materials mattered because of what they were. These were not obscure docket citations buried in a fifty-page brief or a misstated pincite to a secondary source. They were scholarly references in an expert declaration from a scholar retained to address misinformation. The court treated that mismatch as a reliability problem, not a typographical problem.[1]

There is an obvious irony in an AI-misinformation expert using generative AI in a case about the dangers of AI-generated misinformation and then filing hallucinated citations. The court and later commentators noticed it.[2][3] But the filing risk is more important than the irony. Courts do not need to dislike AI to conclude that an expert who signs a declaration containing invented authorities has damaged the very credential the declaration asks the court to rely on.

Why Daubert Made The Error Hard To Cure

The court analyzed the problem through the lens of expert reliability. Under Daubert, the court is not simply checking whether an expert has impressive credentials. It is deciding whether the expert’s testimony is reliable enough to assist the court. A declaration containing hallucinated authorities gives the judge a concrete reason to doubt the expert’s methods, care, and verification practices.[1]

Minnesota’s proposed fix was to file a cleaned-up declaration. Judge Provinzino refused. Her reasoning was severe because the original declaration had already done its damage. Removing the fake citations after exposure would not restore the court’s confidence in the expert’s reliability. The credibility problem attached to the submission process itself.[1]

That is the part litigators should not soften when briefing this case internally. A corrected exhibit may cure a clerical mistake. A corrected expert declaration may cure some immaterial drafting errors. Kohls says that when a sworn expert declaration contains invented academic sources generated by AI, the court may reasonably decide that amendment cannot undo the reliability failure.

The Attorney-Supervision Problem

The most useful paragraph in Kohls for law-firm risk staff is not the one that names ChatGPT-4o. It is the court’s discussion of Rule 11. Judge Provinzino observed that Rule 11’s verification obligation may now require attorneys to ask witnesses whether they used AI to draft declarations and what steps were taken to verify AI-generated content.[1]

That observation changes the practical intake questions for expert work. Before Kohls, a lawyer might have focused on the expert’s qualifications, the factual record supplied, assumptions, compensation, conflicts, and consistency with prior testimony. Those remain necessary. They are no longer sufficient if the expert used a generative tool to draft, revise, summarize, or source the declaration.

The supervising lawyer is not being asked to become the expert. The lawyer is being asked to protect the filing. If a declaration cites articles, cases, reports, data, or quotations, someone has to verify that those authorities exist and say what the declaration says they say. When AI has touched the drafting process, the need to ask becomes harder to avoid.

• Did the expert use any generative-AI tool to draft, edit, summarize, cite-check, translate, or organize the declaration?
• Which passages, citations, quotations, or summaries were generated or materially shaped by the tool?
• Did the expert personally verify each cited authority against a primary source or reliable database?
• Did counsel independently verify citations and quotations that appear in the filed declaration?
• Is there a record of the verification process if the filing is challenged?

Those questions are not ceremonial. They decide who is left cleaning up the record: the expert whose credibility is attacked, the trial team that filed the declaration, the client whose motion record is weakened, and the court that spent time resolving a problem basic verification should have caught.

How Kohls Fits The Hallucinated-Citation Line

Kohls belongs beside the now-familiar hallucinated-citation cases, but it should not be collapsed into them. Mata v. Avianca involved fake cases in a lawyer-filed brief. Park v. Kim and Kruse v. Karlan are part of the same developing sanctions conversation over AI-generated legal authorities. Bracewell’s discussion of Kohls places it in that line while noting the different posture: this was an expert declaration, not simply a lawyer’s memorandum of law.[3]

That difference is not cosmetic. A fake case in a brief threatens counsel’s Rule 11 certification and the court’s legal analysis. A fake academic citation in an expert declaration threatens the admissibility and weight of the expert’s testimony. In Kohls, those concerns merged. The same verification failure created a credibility ruling and prompted a lawyer-supervision warning.

The case also came in an unusually sensitive factual setting. Minnesota was defending a statute directed at deepfakes and election-related misinformation, and its expert submission on misinformation contained AI-generated misinformation of its own.[2] The setting sharpened the court’s concern, but the filing lesson is not limited to election-law cases.

What This Does Not Prove

Kohls should not be used as a shortcut for broader claims the order does not support. It does not prove that AI drafting tools are categorically incompatible with expert work. It does not create, by itself, a freestanding universal disclosure rule requiring every witness to announce every AI interaction. It does not decide the merits of every challenge to deepfake regulation.

It does show that a court may treat unverified AI-generated authorities as more than a drafting mishap. Once a declaration contains fake sources, the issue can become evidentiary, ethical, and procedural at the same time. The expert’s credibility is impaired. Counsel’s verification process is exposed. The client may lose testimony it expected to rely on.

That is why the ruling matters even to lawyers who are tired of theatrical warnings about hallucinations. The problem in Kohls was ordinary in the least flattering sense: someone used a tool that can invent sources, the invented sources were not caught, and the declaration was filed. The sophistication of the subject matter did not save the record.

The Filing-Screening Question

The safest reading of Kohls is practical and narrow. AI-assisted expert drafting is not automatically disqualifying. Unverified AI-generated authorities in a sworn declaration may be. The difference has to be addressed before filing, not after the opposing party finds the problem.

Before an expert declaration is filed, the screening question is simple enough to put on a checklist: has anyone asked the witness whether AI was used, identified what it generated, and verified every authority against a primary or reliable source?

References

1. Order Granting in Part Motion to Exclude Expert Testimony — Kohls v. Ellison, 0:24-cv-03754, U.S. District Court for the District of Minnesota, Jan. 10, 2025, link
2. Judge rebukes Minnesota over AI errors in 'deepfakes' lawsuit, Reuters, Jan. 13, 2025, link
3. The Irony – Using Generative AI in a Case About the Dangers of Generative AI, Bracewell LLP / Westlaw Today, Jan. 30, 2025, link