Why You Should Not Use ChatGPT for Legal Advice

ChatGPT can help you get oriented. It can translate legal jargon into plain English, help you make a list of questions for a lawyer, organize a timeline, or explain the difference between a complaint, an answer, and a motion in general terms. That is not the same thing as using ChatGPT for legal advice.

If the question is “Can I safely ask ChatGPT what to do about my legal problem?” the answer should be no. The danger is not one flaw that a clever prompt can fix. Three risks stack on top of one another: you may expose confidential facts, you may receive fabricated or jurisdictionally wrong law, and the exchange may cross into the unauthorized practice of law without giving you the protections that come with an actual lawyer-client relationship.

That distinction matters for consumers and attorneys in different ways. A consumer who cannot afford counsel may be looking for the only help that feels available. An attorney, by contrast, already works inside professional rules that assign responsibility for competence, confidentiality, candor, communication, fees, and supervision. The American Bar Association made that point directly in Formal Opinion 512, issued in July 2024, when it said existing Model Rules already govern lawyers’ use of generative AI rather than waiting for an entirely new ethics code to appear.[1]

The question has become urgent because chatbot use is no longer fringe behavior. Pew Research Center reported in June 2026 that 49% of U.S. adults use AI chatbots, and 20% use them for medical or advice-seeking purposes; Pew did not isolate legal advice, so that figure should be treated as a directional proxy rather than a legal-specific measurement.[2] Still, it is enough to explain why “chatgpt legal advice” is no longer a curiosity query. People are already asking machines for help with consequential decisions.

Legal Information Is Not Legal Advice

A general explanation describes law without applying it to a person’s facts. Legal advice applies law to a specific situation and tells someone what they should do, avoid, file, argue, disclose, demand, accept, or concede. That shift is where the risk changes.

The safer column still requires care. You should not paste private facts, medical records, criminal allegations, trade secrets, settlement communications, or client materials into a consumer chatbot just because the question sounds general. But the right column is where ChatGPT begins acting like the missing lawyer: applying rules to facts, predicting outcomes, and recommending action without privilege, licensing, malpractice accountability, or a duty of loyalty.

For a consumer, the missing safeguard is not just accuracy. It is the absence of a person who is professionally responsible for asking follow-up questions, spotting jurisdictional traps, protecting confidences, and telling you when your preferred answer is legally weak. For an attorney, the missing safeguard is even harder to excuse: the lawyer remains responsible for the work product even if the first draft came from a model.

Risk One: Confidentiality Can Be Lost Before the Legal Analysis Even Starts

The first problem with ChatGPT legal advice is that the user often has to describe the problem in order to get a useful answer. In law, those details are the valuable material: who said what, when money changed hands, what was admitted in an email, which deadline is approaching, whether someone has a criminal record, or what the client is willing to settle for.

A lawyer receiving that information inside a lawyer-client relationship works under confidentiality duties and privilege doctrines. A general-purpose chatbot does not become your lawyer because you typed facts into it. It does not owe loyalty. It does not check conflicts. It does not carry malpractice insurance for your matter. It does not step into court if the answer harms you.

For consumers, that means a prompt can turn into an unforced disclosure. A tenant might paste a landlord dispute, a worker might describe immigration status and employment facts, or a divorcing spouse might enter financial details. The chatbot may feel private because the conversation is one-on-one on a screen. Legally, that feeling is not the same as privilege.

For attorneys, confidentiality is not optional. ABA Formal Opinion 512 ties generative AI use back to Model Rule 1.6 and related duties, meaning lawyers must understand whether client information is being disclosed, stored, reviewed, or used in ways the client has not authorized.[1] A lawyer who pastes client facts into a consumer tool and then says “the AI did it” has not moved the duty anywhere useful. The supervising attorney, not the chatbot, will be asked why client information left the firm’s control.

This is why architecture matters. A governed legal AI workflow may include contractual protections, access controls, data-retention limits, audit trails, human review, source linking, and matter-level policies. Consumer ChatGPT is not automatically equivalent to that environment. The same interface style can hide very different risk controls.

Risk Two: Hallucinated Law Is Not a Minor Drafting Defect

The most behavior-changing evidence is the hallucination data. In a 2024 Stanford RegLab and Stanford HAI benchmark, GPT-4 hallucinated legal citations in 58% to 82% of tested legal queries.[3] That study tested GPT-4 rather than later models, so it should not be treated as a final measurement of every current or future system. But the size of the range changes the category of the risk. This is not a typo rate. It is a reliability problem large enough to make unsupervised legal reliance unsafe.

Legal hallucination is especially dangerous because the output can look professional. A fabricated case name may be formatted correctly. A non-existent quotation may sound like judicial language. A rule may be real in one jurisdiction and wrong in another. A chatbot can produce a confident answer that fails precisely where legal advice needs to be strongest: authority, jurisdiction, procedural posture, and exceptions.

Legal-specific systems are not immune either. A 2025 Yale Journal of Law and Technology study found that legal-specific retrieval-augmented generation tools hallucinated in 17% to 34% of tested outputs.[4] That comparison cuts both ways. It suggests specialized systems with legal retrieval can perform better than general-purpose chatbots, while also showing why human verification remains necessary even in tools built for legal work.

Courts have already seen the cleanup work. A Boston Bar Journal article reported that, in September 2025, a California court imposed a $10,000 sanction on an attorney after 21 of 23 cited quotations were hallucinated by ChatGPT.[5] The same article cited Damien Charlotin’s public database as tracking 282 U.S. cases and more than 130 international cases involving AI-generated legal errors as of late 2025.[5] Those figures do not mean every lawyer using AI will be sanctioned. They do mean the risk has left the seminar room.

For consumers, the hallucination problem is crueler because there may be no second reviewer. A self-represented litigant may file a motion with fake cases, miss a deadline because a chatbot summarized the wrong rule, or accept a settlement based on a mistaken understanding of remedies. The error may not announce itself. The judge, opposing counsel, or clerk may be the first human expert to see it.

NBC News has reported mixed outcomes for pro se litigants using AI assistance, including some wins and some losses.[6] That is exactly the point to preserve: a small-claims success story does not generalize to complex litigation, and a useful drafting assist does not prove that the tool can supply legal judgment. The lower the stakes and the simpler the procedure, the easier it is to mistake survivable AI help for reliable legal advice.

Why Verification Is Not a Complete Consumer Workaround

People often respond to hallucination risk by saying they will “just verify everything.” That is sensible for a lawyer or trained researcher using AI as a starting point. It is much less realistic for someone who came to ChatGPT because they did not know which sources mattered in the first place.

Verification in law is not only checking whether a case exists. It means checking whether the case is still good law, whether it governs the jurisdiction, whether the quoted passage is holding or dicta, whether later statutes changed the rule, whether procedure has local variations, and whether the facts are close enough to matter. A chatbot can help create a research path, but it cannot make the user competent to evaluate every authority it names.

Risk Three: Legal Advice Can Trigger Unauthorized-Practice Exposure

Unauthorized practice of law, or UPL, is the risk that appears when a non-lawyer provides legal services that a jurisdiction reserves to licensed attorneys. A chatbot is not a person in the ordinary licensing sense, but that does not make the problem disappear. The harder question is who bears responsibility when a product is designed or used to deliver tailored legal conclusions to someone with a real dispute.

That question is now being litigated. In March 2026, Nippon Life filed a lawsuit against OpenAI in the Northern District of Illinois alleging that ChatGPT gave Graciela Dela Torre tailored legal conclusions that contributed to more than 44 allegedly baseless filings and $10.3 million in claimed damages.[7] Those are allegations in an early-stage complaint, not proven facts and not a court ruling that ChatGPT committed UPL. Their significance is narrower but still important: a major plaintiff has framed chatbot legal guidance as a UPL and liability problem, not merely a user-misuse problem.

OpenAI’s own policy posture is part of that debate. Stanford CodeX has argued that OpenAI’s October 2024 terms-of-service update prohibiting “tailored legal advice” should be understood as an admission of foreseeable risk rather than a complete defense to liability.[8] That is an argument, not a final judicial conclusion. But it captures the basic tension in consumer-facing AI: the product can feel conversational and helpful while the legal consequences are pushed back onto the user who relied on it.

For consumers, UPL risk matters because licensing rules are supposed to protect the public. A licensed lawyer can be disciplined, sued, disqualified, or reported to a bar authority. A chatbot cannot be cross-examined about its professional judgment. It cannot refund a fee because it failed to investigate. It cannot recognize that it should decline representation because of a conflict. When the advice is wrong, the injured person may discover that the tool was never accountable in the way a lawyer would have been.

For attorneys, the issue is not that AI assistance is forbidden. It is that a lawyer cannot launder legal judgment through a model. If a lawyer uses ChatGPT to draft a brief, evaluate a claim, summarize discovery, or recommend a settlement position, the lawyer must still satisfy duties of competence, candor, communication, confidentiality, reasonable fees, and supervision. Formal Opinion 512 points lawyers back to those duties rather than creating an AI exception to them.[1]

What a Lawyer or Governed Legal AI Workflow Adds

The comparison is not ChatGPT versus nothing. For many people, unfortunately, it may feel that way. But the legal standard for advice is built around protections that a general-purpose chatbot does not supply.

• Confidentiality: a lawyer has duties that restrict disclosure and misuse of client information.
• Competence: a lawyer must know the relevant law or become competent through reasonable preparation.
• Jurisdictional judgment: a lawyer must account for local statutes, rules, deadlines, judges, and procedure.
• Accountability: a lawyer can face discipline, malpractice claims, sanctions, fee disputes, and reputational consequences.
• Supervision: in a law practice, non-lawyer assistance and technology use must be reviewed by responsible attorneys.

A governed AI workflow tries to preserve those protections while using software for limited tasks. That may mean using legal-specific tools, restricting data entry, verifying every cited authority, keeping audit logs, training users, requiring partner review for court filings, and defining which tasks are off-limits. Attorneys looking for that narrower implementation question can use resources such as What Lawyers Must Know Before Using ChatGPT or a dedicated ethics workflow for free AI tools and confidentiality. The point is not to make consumer ChatGPT into a lawyer. It is to keep AI assistance inside a professional system that already has duties, reviewers, and consequences.

That system can still fail. The Yale hallucination figures for legal-specific RAG tools make clear that legal AI branding is not a warranty of correctness.[4] But the governed environment at least gives the firm or legal department a place to assign responsibility: who may use the tool, which data may enter it, which outputs require source verification, and who signs off before anything reaches a client, court, or opposing party.

What Consumers Can Safely Ask ChatGPT

There are useful, lower-risk ways to use ChatGPT around a legal problem. They stay on the orientation side of the line and avoid private facts where possible.

• Ask for plain-English definitions of general legal terms.
• Ask for a checklist of documents to gather before meeting a lawyer.
• Ask for questions to bring to a legal-aid clinic or attorney consultation.
• Ask it to organize a non-confidential timeline that you have already anonymized.
• Ask where to look for official court forms, legal-aid organizations, or government self-help pages, then verify those sources directly.

The line to avoid is a prompt that asks the system to decide your rights, select your strategy, draft a filing for your specific dispute, predict what a judge will do, or tell you whether to accept, reject, disclose, file, threaten, or settle. Those are legal-advice functions. For a more affirmative list of limited tasks, see which legal tasks ChatGPT can handle safely.

If money, housing, immigration status, custody, criminal exposure, employment, debt collection, medical injury, business ownership, or a court deadline is involved, the safer route is a licensed attorney, a legal-aid office, a court self-help center, a bar referral service, or a properly supervised clinic. Those resources may still be imperfect and hard to access. They at least operate closer to the system that can protect confidentiality and assign responsibility.

What Attorneys Should Do Instead

Lawyers do not need a rule that says “never use AI” to know that consumer ChatGPT is the wrong place for unreviewed legal advice. The safer question is operational: what workflow would let a lawyer use AI without surrendering competence, confidentiality, candor, or supervision?

• Use firm-approved tools with written data-handling terms before entering client information.
• Treat AI output as unverified work product until a lawyer checks every legal proposition and citation.
• Require jurisdiction-specific review before advice, filings, or client communications leave the firm.
• Disclose AI use to clients when required by the engagement, the jurisdiction, the fee arrangement, or the nature of the work.
• Document supervision rules for associates, staff, contractors, and vendors using generative AI.

Those controls are not busywork. They are the mechanism by which responsibility stays with the lawyer instead of evaporating into a chat window. A firm drafting policy can go deeper on closing the governance gap for AI legal research, and lawyers practicing in California should check jurisdiction-specific materials such as California State Bar AI ethics guidance.

The sanction examples also show why courts will not be impressed by a workflow that ends at “I asked ChatGPT.” If a brief contains fake quotations, the court’s problem is not that the lawyer used modern software. It is that the lawyer filed unreliable material under a professional signature.

The Routing Decision

ChatGPT is not unsafe for legal advice because it is useless. It is unsafe because legal advice is a regulated, confidential, accountable act, and a general-purpose chatbot does not supply those safeguards. The confidentiality risk begins as soon as sensitive facts enter the prompt. The hallucination risk remains even when the answer sounds polished. The UPL and liability risk appears when tailored legal conclusions are delivered without the structure that licensing is supposed to provide.

Consumers should use ChatGPT, if at all, for general orientation and preparation: vocabulary, questions to ask, document organization, and pointers to official resources that can be independently checked. Actual legal problems should go to licensed counsel, legal-aid organizations, court self-help centers, or properly supervised clinics.

Attorneys should not use consumer ChatGPT as a substitute for legal judgment. AI-assisted legal work belongs inside governed workflows, firm policies, source verification, client-confidentiality controls, and jurisdiction-specific ethics review before it touches client matters, court filings, or legal advice.

References

1. ABA issues first ethics guidance on a lawyer’s use of AI tools, American Bar Association, July 2024
2. Americans and AI in 2026: Chatbots, smart devices and views on impact, Pew Research Center, June 17, 2026
3. AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries, Stanford HAI, 2024
4. Yale JELS legal-specific RAG hallucination study, Yale Institution for Social and Policy Studies, 2025
5. ChatGPT Is Not a Lawyer: Using Generative AI Responsibly and Ethically in Law, Boston Bar Journal
6. AI Chatbots Are Helping People Represent Themselves in Court. But Are They Any Good?, NBC News
7. When Is a Settlement Not a Settlement? AI, ABA Law Technology Today, 2026
8. Designed to Cross: Why Nippon Life v. OpenAI Is a Product Liability Case, Stanford CodeX, March 7, 2026