When Prosecutors Hallucinate: AI Citations in Criminal Cases

The first thing that matters is not the software. It is the filing.

In Nevada County, California, District Attorney Jesse Wilson acknowledged that his office had “recently used artificial intelligence in preparing a filing, which resulted in an inaccurate citation.” The office withdrew the filing. Wilson also said, “At no time was there any intent to mislead the court,” and said the office had implemented new AI training and policies after the incident.[1]

That posture is important. The public record, as reported, contains an admission that AI was used in preparing a criminal filing and that an inaccurate citation resulted. It also contains the prosecutor’s denial of intent to mislead. Those two facts can coexist. A fabricated citation does not need bad faith to do damage in a criminal case.

Civil Rights Corps treated the Nevada County matter as more than a clerical embarrassment. It identified three separate criminal cases with what it described as AI-typical errors, including nonexistent quotations and misread holdings, and petitioned the California Supreme Court. The organization framed prosecutorial reliance on AI-generated authority as “an existential threat to the due process rights of criminal defendants.” As of the November 2025 reporting available for this record, that petition was pending.[1]

The unresolved factual dispute matters. The district attorney characterized the problem as a single incident. Civil Rights Corps alleged a broader pattern across three cases. The difference changes how severe the episode looks, but it does not change the immediate procedural problem: the state filed authority into a criminal case, the authority was wrong, and the remedy described in the public record was withdrawal, training, and policy revision rather than sanctions.

The burden sat on the side with filing power

In a civil AI-citation scandal, the usual story is reputational: a lawyer embarrasses herself, a judge issues an order to show cause, the client may pay for the mess, and the bar may take interest. In a criminal case, the same defect enters a different machine. A false case citation can support custody, preserve leverage, defeat suppression, narrow discovery, or push a defendant toward a plea before anyone has the time to determine that the case does not exist.

The asymmetry is built into the process. The prosecutor has the power to file. The defense often has the burden of answering on a short schedule. The court may see a citation in an official filing and assume it resolves to real authority. If the defense suspects an AI-generated error, it usually cannot depose the prosecutor’s drafting process, inspect internal materials, review office policies, or compel a clean explanation of who checked what unless a court first treats the suspicion as worth litigating.

That is why intent is not the center of the risk analysis. Intent may matter for discipline, sanctions, and credibility. It does not answer the institutional question. The state had the burden to verify its own authority before filing it. The defendant had the consequence of the filing. The court had to decide the motion in front of it. A hallucinated citation can move through that sequence even when nobody set out to deceive anyone.

The second confirmed prosecutor-side example came from DOJ

The Nevada County case is not the only confirmed U.S. prosecutor-side episode now in the record. In July 2026, a Department of Justice Assistant U.S. Attorney cited a nonexistent Sixth Circuit case in an immigration detention filing in Daghra v. Hinkley. Chief Judge Hala Jarbou admonished the government lawyer but did not impose sanctions.[2]

The Michigan case is not a perfect mirror of the Nevada County criminal prosecution. It arose in an immigration detention posture, and the source record describes the issue as an alleged AI-use fake-citation problem. But for defense and civil-rights lawyers, the shared feature is the one that matters: a government lawyer invoked nonexistent appellate authority in a liberty-adjacent filing, and the court’s public response stopped at admonishment.

Chief Judge Jarbou’s warning was direct: “it should be obvious that any attorney who uses AI must scrupulously review its work product.”[2] That sentence is useful because it states the professional baseline without requiring a special AI rule. A lawyer who files the document owns the document. The machine does not become an associate of record, and it does not absorb the duty of candor.

Rare does not mean harmless

The available tracking data does not support a claim that prosecutorial AI hallucination is common. Damien Charlotin’s AI Hallucination Cases Database listed 1,811 global cases as of July 29, 2026, including 1,252 U.S. cases and only five documented instances involving prosecutors using AI.[3]

That makes prosecutor cases roughly 0.4% of the U.S. cases in the database. The number should be handled carefully. A third-party database is not a complete census of every defective filing, and inclusion depends on detection, reporting, and classification choices. It is still a signal. In a criminal or detention-adjacent setting, a small number of confirmed government-side examples can matter more than a larger number of private-lawyer embarrassments because the state’s filing carries coercive force.

The remedy gap is the real issue

Once a fabricated citation is found, the narrow correction is obvious: withdraw the filing, correct the authority, notify the court, and train the lawyers. Those steps may prevent the same office from repeating the same mistake next week. They do not answer what happens to the defendant whose motion was shaped by a false statement of law, or whose lawyer had to spend limited time proving that the government’s case did not exist.

Neither confirmed matter described here produced sanctions. Nevada County withdrew the filing and announced internal changes.[1] In Daghra, the court admonished the government lawyer but declined to sanction.[2] That leaves defense counsel with a practical problem: the best available relief may be case-specific and discretionary, even when the underlying defect is easy to state.

A due-process argument can be framed several ways. Counsel may argue that reliance on fabricated authority deprived the defendant of a fair adversarial process, that the prosecution’s verification failure burdened the defense, that the court should require disclosure about AI use in preparing the challenged filing, or that the tainted motion should be stricken rather than merely corrected. The strength of any remedy will depend on timing, prejudice, the role the false authority played, and whether the filing affected custody, plea pressure, discovery, suppression, or sentencing exposure.

The harder question is proof. A defense lawyer can show that a citation is fake. Proving how it got there is different. Without disclosure, the defense may have only surface signals: a case name that does not appear in ordinary databases, a quotation that sounds judicial but cannot be found, a holding that overstates a real case, a parenthetical that misreads the procedural posture, or a citation format that looks plausible while failing at every point of verification.

Detection signals counsel should treat as litigation facts

• A cited case does not resolve in the reporter, Westlaw, Lexis, Google Scholar, or the issuing court’s own database.
• A quotation appears nowhere in the cited opinion, even though the quoted language is central to the prosecution’s argument.
• The cited case exists, but the holding is described as broader than the opinion supports.
• The parenthetical states a criminal-law rule while the actual case concerns a different procedural setting.
• Multiple authorities in the same section show the same pattern: real-looking captions, confident quotations, and no verifiable source trail.

Those signals should be preserved in the record, not just corrected in the lawyer’s notes. Attach database search results if local practice permits. Identify the exact sentence that depends on the false authority. Ask the court to require the government to state whether AI was used in drafting or cite-checking the filing. If the court will not order disclosure, make that refusal part of the appellate record.

Professional discipline is a secondary layer, not a substitute remedy

The professional-risk lesson for prosecutors is plain enough after Daghra. A court has now said that any attorney using AI must scrupulously review its work product.[2] That warning applies with special force to government lawyers, who are not just advocates but public officials exercising charging and litigation power.

Still, discipline should not be overstated from the present record. The known court responses described here did not impose sanctions. The Nevada County public record, as reported, includes a denial of intent to mislead and remedial training.[1] The Michigan order admonished but did not sanction.[2] Those facts do not eliminate bar exposure, but they limit what can responsibly be said today: the professional duty to verify AI-assisted work is serious; liability in these two matters has not been established in the materials available here.

For offices trying to manage risk, the useful question is not whether AI is forbidden. It is whether the office can prove that every filed authority was checked by a lawyer who understood the motion, the cited case, and the consequence of being wrong. A policy that says “review AI output” is weaker than a record showing who cite-checked the brief, what sources were checked, and how corrections were made before filing.

Why the California Supreme Court petition matters

Civil Rights Corps’ California Supreme Court petition is the natural place for the unresolved question to concentrate because it asks what ordinary motion practice has not yet supplied: a due-process remedy for prosecutorial AI hallucination. The petition was supported by an amicus brief from 22 scholars and advocates, according to the available reporting.[1]

The petition’s importance does not depend on proving that prosecutor hallucinations are widespread. The available data points the other way: they are rare in the tracked universe.[3] The importance comes from the identity of the filer and the procedural setting. When the state files fabricated authority, the defense may not be able to discover the drafting process, the court may not impose sanctions, and the corrected filing may arrive only after the defense has spent time and leverage on an error it did not create.

For now, the record gives practitioners three usable propositions. First, there are at least two confirmed U.S. prosecutor-side AI-citation matters in the public record: Nevada County and the DOJ AUSA case in Michigan.[1][2] Second, the dominant judicial response so far has been admonishment or correction, not sanctions.[1][2] Third, defense counsel should verify prosecution authority aggressively, especially when a filing relies on quotations, holdings, or case names that appear only inside the government’s brief.

The boundary is just as important as the warning. This is a confirmed but rare pattern, pending further court action and better reporting. Every status claim, especially the status of the California Supreme Court petition, should be checked against primary docket sources before publication or litigation use.

References

1. California prosecutors' office used AI to file inaccurate motion in criminal case, The Guardian, Nov 2025
2. DOJ Called Out by Judge for Alleged AI Use, Fake Case Citation, Bloomberg Law, July 2026
3. AI Hallucination Cases Database, damiencharlotin.com