What Are We Going to Do About AI Slop?

Remember when parties developed unrealistic expectations all by themselves?  Now they have AI to help.

These days, mediators may have to work through expectations that have hardened around an AI-generated assessment.  Lawyers may have to explain why the AI analysis is wrong, incomplete, or wildly overconfident.  And imagine the client’s surprise when the other side may get to read their AI chats – embarrassing information and all.

The problem of AI slop is bigger than just hallucinated cases.  People can use AI in ways that omit inconvenient facts, reinforce dubious assumptions, and expose confidential information.

When unreliable AI-generated material enters a dispute, the problems don’t stop with the person who generated it.  Lawyers, counterpart parties, mediators, courts, agencies, and others may have to spend lots of time figuring out what is real and what is slop.

There’s Plenty of AI Slop to Go Around

I started thinking about this problem when the head of a mediation unit in a government agency contacted me because she was at her wit’s end.  The mediators on her team were regularly struggling when self-represented parties (SRPs) came to mediation relying on AI.

Since SRPs often can’t afford lawyers, it’s completely understandable that they would turn to AI.  But that can make things harder for everyone.

This situation prompted me to develop two guides to help SRPs use AI more wisely.  One is a concise, graphic guide designed particularly for people with limited literacy.  The other provides more detail and focuses on using AI in mediation.

I realized that this isn’t just a problem about SRPs’ use of AI.  Pretty much everybody – including legally represented parties, lawyers, and mediators – can misuse AI and create problems for themselves and others.

So just telling SRPs to be more careful isn’t enough.

We Need Better AI Hygiene

Good AI hygiene involves habits such as verifying legal authorities with reliable sources, protecting confidential and sensitive information, testing assumptions, and – most important – exercising independent judgment instead of outsourcing important decisions to a bot.

The goal is more benefit and less slop.

Mediators and lawyers can help without becoming AI cops.  Instead of asking whether a client used AI, they can probe their reasoning:  What facts is that assessment based on?  What information might be missing?  What assumptions are involved?

We Need a Public Legal Health Campaign

The more I thought about it, the more this looks like a public legal health problem.

Bad AI use imposes costs on everyone who has to deal with AI slop.  We have a systemic problem because courts, lawyers, mediators, and counterpart parties will increasingly have to spend time separating reliable information from slop.

A public legal-health initiative would try to prevent some of those problems before they cause trouble for others.

Education is part of the answer.  Courts, agencies, mediation programs, bar associations, law schools, lawyers, and mediators can teach and model good AI hygiene.

Law schools have an especially important role to play.  All law students need practical training to use AI competently – not only to get its benefits and avoid problems in their own work, but also to evaluate AI-generated material they receive and counsel clients who use AI.

AI competence is becoming part of basic professional competence, so learning it shouldn’t be optional.  Law schools should develop school-wide policies that help students learn effectively both with and without AI, with structured AI use that can expand as students progress through law school.

Good AI hygiene and a broader public legal-health effort won’t be enough to prevent all bad AI output.  But better education, professional practices, AI tools, and institutional responses could reduce the amount of unreliable material circulating through dispute-resolution systems.

None of this solves the underlying access-to-justice problems that lead many people to represent themselves.  But it can address problems that arise when SRPs – and everybody else – misuse AI in disputes.

So What Are We Gonna Do About the Slop?

AI isn’t going away.  Neither are hallucinations, overconfidence, careless prompts, or people who believe what a bot tells them because it sounds authoritative.

We can, however, get better at dealing with these problems.  A broad public legal-health effort could help.

I develop these ideas in more detail in a forthcoming article in the Dutch-Flemish Journal of Mediation and Conflict Management.  I’ll also discuss them at the AALS-ADR Works-in-Progress Conference at the University of Missouri on September 25.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.