Artificial Intelligence in the Courtroom: Risks and Realities

Episode 6: In this podcast episode, Louisville family law attorney Louis Winner and his legal team discuss the problematic rise of artificial intelligence within the legal field. The group discusses how unrepresented litigants and clients frequently use AI to generate fraudulent case law, non-existent rules, and irrelevant legal arguments that clutter the court system. While AI can assist with basic document organization, it lacks the human intuition, emotional intelligence, and advocacy skills necessary for effective trial work. Louis and his colleagues express concern that the technology provides users with a false sense of confidence in meritless positions. They warn that over-reliance on these tools often leads to increased legal costs and potential judicial sanctions for those who fail to verify AI-generated content.

The firm’s primary focus is family law. This includes divorce, custody, business valuations, alimony (referred to as “maintenance” in Kentucky) and high-income child support cases.

Key Themes

• Why You Shouldn’t Hire AI to be Your Legal Counsel
• Issues Caused by Clients Using AI to Draft Documents
• Clients Using AI for Their Own Legal Research
• The Risk of Using AI to Develop Questions for Cross Examination
• Should Attorneys Use AI to Draft Legal Documents?
• How Are the Courts Handling the AI Dilemma?
• Benefits of Using AI for Legal Matters
• Will AI Ever Replace Actual Attorneys?

Episode Transcript

Why You Shouldn’t Hire AI to be Your Legal Counsel

Louis Winner:

Hi, this is Louis Winner. I'm here with two members of my law firm, attorney Anna Hudson and paralegal Cassandra Mastropaolo. We're going to talk about something that has been all over the news recently, artificial intelligence and the effect of this on law and in the courtroom and what happens. Over the last, I don't know, I'd say about a year, AI has just exploded in the legal field and it has good points. It has bad points. I'm going to talk about the bad points first because that's more fun. So, one of the things we've noticed, there are a lot of pro se litigants out there at this point. Pro se litigants are litigants who do not have attorneys and they represent themselves. And what they have done, understandably so, is they've gone to AI and used that in order to file documents with the court, things we call pleadings.

And that sounds great, but there's some problems with it. The information that we see coming from AI, from pro se litigants, isn't stuff that always makes sense. In addition to it not making sense, we are seeing just a plethora of arguments and pleadings that have truly nothing to do with the facts of someone's case. So we have some cases in the court of appeals right now, which is the second highest court in Kentucky, in our state court level. And we have someone on the other side of one of our cases. We have several cases going on with pro se litigants who seemingly every week we get a new filing and it is obviously generated by AI. It takes us all of about 10 seconds to realize that. And we then have to respond to these things. So we get these documents that'll be a neat legal issue and wow, we get that issue. It's really cool.

The problem is it has absolutely nothing to do with his or her case that we're working on. So oftentimes I'll take something to Anna and Cassandra and say, "Hey, take a look at this and start drafting up the response,” and tell me what you guys have to do at that point.

Anna Hudson:

I mean, God, it's just you spend so much time trying to figure out even where to start because not only are you trying to figure out the logical arguments that you normally do with a response, but you're trying to figure out what AI thinks is a logical argument. And then you have the irrelevant facts that AI somehow will weave in a way that makes absolutely no sense. You're trying to decipher what are even applicable facts to these insane arguments.

Cassandra Mastropaolo:

Yeah. And a lot of times too, I don't think that people understand the prompts that they are giving AI are already not rooted in any kind of legal basis because as a pro se litigant, they don't know right from wrong, they don't know good legal arguments to make. So the prompts that they're feeding into this artificial intelligence to begin with aren't really strong jumping off points for that AI to hold onto and then create. You're kind of creating nothing out of nothing at the end of the day.

Anna Hudson:

Yeah. AI never says you don't have a case here. AI makes a case every time and they don't understand that.

Louis Winner:

That's a great point. AI is doing what you tell it to do. Hey, you need a case on this? Well, we'll make up a case on this. And sometimes when I say make up a case on this, we're literally making up a case. So we got something actually from one of our clients recently who said, "Hey, you need to look at this specific rule." And I know we're all kind of laughing for a second and we're laughing because I got it and I looked at the document and I know most of the rules top of my head. Of course, I don't know all of them. So I'm like, "You know what? Give me that rule book again." And I look through it, I'm like, just as I thought, said rule doesn't exist. So when the client who drafted me this correspondence, with the use of AI, sent it to me and saying, "Hey, you need to look at rule X, Y, Z." Well, rule X, Y, Z was literally made up by AI.

So you have to then tell your client, I'm sorry, what you wanted to get done is a literal impossibility because that rule that you were told exists doesn't exist at all.

Cassandra Mastropaolo:

I think it really instills a false sense of confidence in not only pro se litigants, but also just litigants in general who are prompting AI to give them a position or, hey, do I have a legal standing to do X, Y, and Z? And ChatGPT is telling them yes, when in reality they don't have a case for anything at all. And this confidence that comes with this and this false sense of elevated, I'm rightness that comes with just being told, yes, you're right and here's how you do it when in fact that's not the case at all.

Issues Caused by Clients Using AI to Draft Documents

Louis Winner:

I can't agree more with that. So that brings us to the next issue. Our clients are also sending us correspondence. Sometimes they are drafting their own documents through AI and we get them. And it takes us a considerable amount of time now to go through these documents we're getting to explain to the client that, hey, it looks really pretty, it looks really nice, but it has absolutely no use in what we're doing and it's wrong. Go ahead.

Cassandra Mastropaolo:

It's wrong. And also what would be more helpful for us, I was just thinking about this the other night, we have a case right now with about a million underlying documents and things to go through, a huge volume of which we just got in, and the client thought it would be helpful to feed all of that into AI and put together a summary. What would really have been helpful is if they could have told us what underlying documents to look at as opposed to creating a, what was it? I think 95 single spaced document for us to then read in addition to all of the underlying documents to go through.

Louis Winner:

Right. And they also like to send that to me around 11:00 PM the day before a meeting, so I will get a 95-page, single spaced, AI generated document that I need to read to understand it. And as I'm reading a lot of times, one, again, it's so obvious when something is AI generated. And as you're reading it, you're like, "No, that's not right. No, no, that's not right. This is wrong." And by the time you're on page 42, you're like, "This is a waste of my time." And instead of saving the client money, it's not costing the client money.

Cassandra Mastropaolo:

Oh, that's so for sure. An extraordinary amount of money.

Louis Winner:

Right. And sometimes I'm asking either Cassandra or Anna to summarize this, tell me what's going on here. Does this make sense? And then they're stuck reading these very lengthy, AI generated documents that are result-oriented. And the client of course wants result X or Y and has set those prompts into AI to get that result that he or she wants.

Cassandra Mastropaolo:

It is kind of funny, the circular nature of it too, because when I get things like that to review, all it makes me want to do is put it into ChatGPT and say, summarize this for me and figure out how do you even begin to delineate from those massive documents.

Clients Using AI for Their Own Legal Research

Louis Winner:

So, we have that from our clients that we're dealing with. This is new and that I've been doing this about 25 years and 10 years ago there was no AI. So you had clients who would say, Hey, I want to have this happen in this way. But now I'm getting, they're citing the cases and that the cases they cite to sometimes do exist, but they're not particularly applicable or there's stuff that I know off the top of my head.

A lot of times clients want the other side to pay their attorney's fees, not an uncommon thing. Nobody actually wants to pay attorney's fees. I understand that. And in Kentucky and some other states as well, there are methods for the other side to pay your fees, but it's so fact-based that AI can understand that. So clients want to focus on their one little thing there, but it's not giving you a good representation of a likely outcome.

Cassandra Mastropaolo:

And it's also always going to be confirmatory. If you tell AI or ChatGPT, this person should have to pay my attorney's fees because of X, Y, and Z and say that in a way that I can send it to my attorney, well, again, it just creates that false sense of rightness.

Anna Hudson:

Well, I think it's also clients aren't lawyers, so they're usually not asking ChatGPT, is there a case for this? Is there this? They're saying, "This is what I have. Tell me what it is." Not to say that Chat would come in with an open mind and give them the best advice, but clients aren't asking for that. And they do that with us too. They want us to tell them what they want to hear, but we have done the work and we know to tell them this is what actually is the law and what's going to be helpful for you in a way that AI never will.

Cassandra Mastropaolo:

And what's realistic.

Anna Hudson:

Right.

Cassandra Mastropaolo:

They’re not getting realistic outcomes from asking ChatGPT or Claude or any of these other AI what to do.

Anna Hudson:

Yeah.

The Risk of Using AI to Develop Questions for Cross Examination

Louis Winner:

So one other thing I've seen with AI, some attorneys have done, is they'll take a document, let's say a custody evaluation, and they'll put it in AI and they'll say prompt like, need cross-examination questions and it'll spit out a hundred cross-examination questions. If you think that's the same as going through that document and writing the questions and learning it and feeling it, you're out of your mind.

Cassandra Mastropaolo:

For sure.

Anna Hudson:

Absolutely. Because I mean, cross-examination is about what is said on direct for the most part. So you need to be able to know what's in that report and wait for the buzzwords of what they're going to say during their direct examination. You can pre-write cross questions obviously, but if you're not in that report and learning it and knowing it inside and out, you're not going to be able to effectively cross-examine someone.

Louis Winner:

I can't agree more. So when we prepare for a case, we will often write out some cross-examination questions, but a lot of cross is how I feel at that point, my gut. Hey, I just heard this. I'm changing it just from what I heard the witness say on the stand. And if I know the document and I've read the 500-page report, then I can do it. When I have just relied upon a machine to tell me what to do, I can't. You can't pivot like that when you don't really know it.

Cassandra Mastropaolo:

Yeah, absolutely. And I think it's interesting that you said several times it's your gut instinct, you don't feel it. It's those emotive components that a robot will never have that when you're reading a report and you're feeling for your client and you're feeling for your position and you get that either incensed or you find something that you just can't believe that you can't wait to hit this person on cross-examination over, you don't experience that same thing. And I would have to imagine you lose some of the sense of advocacy when you can't feel that kind of when you take the feeling out of it.

Should Attorneys Use AI to Draft Legal Documents?

Louis Winner:

Yeah, I agree. The other big thing that we have seen across the country is attorneys using AI to assist them in writing briefs. Okay, maybe you can do that here or there. Here's the problem. Again, AI is not so reliable at times, and the problem across many states is that AI is literally making up case law and attorneys aren't checking. So they are citing to cases that do not exist and they are getting sanctioned time after time after time because they're not even. I mean, they're so lazy. I mean, let's compound the lazy here. Okay? So we get into AI, we write the thing in AI. We don't really write it, but the computers writes it, and then we don't even bother to check it. And then you get called out by a judge who does go check it and it's not there.

Anna, do you remember a case once when we were working on I think something in the, I don't know if it was in the court of appeals or the trial court, and I'm like, "Go check all of the cases that were there.” What happened? Do you remember?"

Anna Hudson:

Yeah, it was the court of appeals. The holding was correct, but they had a quote from the case, and they said they didn't mean to put it in quotes later on, but they had a quote from the case that was not in the case.

Louis Winner:

Right, right. I mean, we all miss quote here and there when we have a typo, but this wasn't a typo.

Anna Hudson:

No, it was very clear. They had changed the quote enough and they'd gone back in to put quotes in it. It was very obvious that it was intentionally done. And had I not checked it, I mean, I never would've known. And luckily, it was an applicable holding and it was correct, but it was not a quote from the case. So then you're screaming to the court of appeals, "Hey, I used AI for this," because that wouldn't happen if you had written it yourself. I just think that it's really embarrassing, honestly.

Louis Winner:

Honestly, I can't agree more when you found that. We mentioned it in our responsive pleading, and we did it nicely because the attorney gave the correct holding as you indicated, but we needed to say, "Hey, this quote doesn't exist here." And that was important to point out. But in general, not just that case, we've seen it in media all over the country, that attorneys are just incredibly lazy.

Now, yesterday, I read something that I hadn't read before, which was a judge in Oklahoma. The judge in Oklahoma wrote an opinion. Cass is laughing because she read it too. The judge wrote an opinion and that opinion was AI-generated, and the judge's opinion contained fake case law too.

Anna Hudson:

Oh, God.

Louis Winner:

Right, right.

Anna Hudson:

Had the case law been cited by attorneys and they just took their word on it, or did they just generate their own?

Louis Winner:

I'm not sure. I read it yesterday. I didn't go back and read the cases themselves and the briefs that were filed. They just said that the judge literally cited law that did not exist in reality. And you're just like, "This is the judge doing it." We can't have a system of one computer talking to another computer and getting these fictitious documents at the end of the day.

How Are the Courts Handling the AI Dilemma?

Cassandra Mastropaolo:

No, and correct me if I'm wrong, but we haven't seen any kind of standards implemented or any restrictions or any even case law on pro se use or attorney usage of AI in any court. Am I correct in saying that?

Louis Winner:

I think you're correct. I haven't. I don't know of any. And a real question becomes, is the use of AI practicing law without a license when AI is literally practicing law and telling you how to practice law? I think it's something the Bar Association should take a look at and decide what's going on here. We had a pro se litigant in a case who had her iPhone out and was having it listen to what was going on, and it was generating questions for her to ask on either cross or direct examination.

Anna Hudson:

And objections.

Louis Winner:

And objections.

Anna Hudson:

It would tell her to object. She would object, and then we'd have to wait for it to generate what the actual objection was.

Louis Winner:

Right. I think that's the practice of law. I think it's the practice of law, and I think that's a real problem, and it wasn't giving correct information. I mean, and that's even the worst part about it. I brought it to the court's attention, you guys were with me, and the judge allowed it to go on. I'm not sure if that was the right or wrong move, but we saw this kind of craziness live in a courtroom of a computer really representing this person.

Cassandra Mastropaolo:

Absolutely. And I don't think the judges know what to do with that yet. I can't imagine. I mean, think, for example, of a case with two pro se litigants, neither of them. There's no counsel in the room to call it out to even bring it to the court's attention. So unless there is some kind of standard or some kind of something that these judges know to implement or to look towards, I don't know if they even know where to go with it.

Louis Winner:

Good point. So you imagine that you've got two pro se litigants. Again, nothing wrong with that, and people's finances sometimes need that to happen.

Cassandra Mastropaolo:

Absolutely.

Louis Winner:

That's okay. But each of them having their iPhones out, telling them what to argue one after another. So you have two ChatGPTs or AIs of different kind, Claude, whatever it is, each other. And hopefully there's actually a judge there who doesn't have his out there or hers out there too.

Anna Hudson:

Well, I mean, the crazy thing is they could both put the exact same thing into their respective AI things, and it's going to generate two different things. There's not even an objective truth in AI. It's telling you what you want to hear.

Benefits of Using AI for Legal Matters

Louis Winner:

Right, right. So benefits, there are some benefits today. I don't want to pretend like it's a bad thing completely. It can go through a large batch of documents and highlight things. We have used it to say, "Hey, we need to find where this word is quickly or this phrase is," and you've been able to find that for me in two seconds, Cassandra.

Cassandra Mastropaolo:

It has made some search functions, some batching functions incredibly easy, especially in these huge documents that we get from clients or batches of discovery that we get, it does make that easier.

Louis Winner:

Right. Other good points of AI. Sometimes we can put something in there and it'll perhaps think of something we haven't thought of. So it's not horrible, and I'm not saying don't use it. I'm saying be cautious in how you're using it. And I'm saying that we have had not just the other side on cases, but our own clients who are sometimes drafting books for us now. I met with a client who made a little chart for me, once, explaining proper objections to things and why this document wasn't appropriate that the other side had filed.

I reviewed that document with the client and it was all wrong. And it was all wrong because those were documents that you make in objections to courts as opposed to a response where the rules of evidence aren't really applicable in the same manner that when you're in court and Chat didn't understand that or whatever AI was used, and the client didn't understand that at all.

Will AI Ever Replace Actual Attorneys?

Cassandra Mastropaolo:

I’m so curious too, if AI is going in the direction that it seems to be going, that everybody seems to be using it as if it's already in that place, does it get to a point where it is a viable, useful resource in that way where it can pull correct, it doesn't hallucinate case law, it doesn't just tell you what you want to hear, that maybe it can be a little bit more discerning? I mean, does it ever get to that point?

Louis Winner:

Anna, what do you think?

Anna Hudson:

I mean, it kind of scares me, I got to be honest. But just where AI was two years ago versus where it is now, it's only going to get better. So I imagine that it will get to a point where it's going to be more accurate, but I think, I mean, what you said earlier about emotion, you can't properly advocate for something as a machine. I just don't think you can. You lose that sense of advocacy that is so crucial to our profession. So I don't think that it's ever going to get to a point where it can actually do everything that everyone wants it to. I mean, it's just insane, but I do think it will get better and it has gotten so much better. I mean, I know that even LexisNexis has an AI function on it that I had whenever I was in law school, and it doesn't hallucinate cases.

It only pulls from cases that are in reporters or unpublished cases, but only from cases that are on LexisNexis.

Louis Winner:

Okay. So I think AI does get to the point that only real cases are generated. I don't think we're so far away from that. I think that'll happen sooner than later. But as Anna, what you said is that it's still a machine and there's no way we get to the point that this AI machine is going to be able to replace human thought, emotion, and intuition and gut, which are so important.

And what's hard to explain sometimes is we go to court always in person 99% of the time, and one of the reasons we do that is that we make physical eye contact with the judge. We see the judge's demeanor, how they look, how they're responding to an argument, and that changes our argument. It changes how we present things sometimes, and judge could be in a good mood, bad mood. She may be happy about something, and you miss those subtle cues, which are so important, those micro expressions that we all do.

Anna Hudson:

Oh for sure.

Louis Winner:

So AI, yeah, it will change and it will get better, but I don't think it ever gets to the point of you can really rely upon this to do what we do.

Anna Hudson:

Unfortunately, I don't think that people will ever understand that. I mean, they don't understand it now. Even we can point and be like, "That's not even a real case you just gave us," and they're still like, "No, it knows all."

Louis Winner:

Right, right. We do get that feedback from clients of like, "No, no, no, you're wrong. The computer told me. You don't understand." All right, well, thank you all. It was a fun conversation and thanks.

Cassandra Mastropaolo:

Yeah, thank you guys.

Anna Hudson:

Thank you.

For more information about Louis Winner and his family law practice visit: www.LouisvilleFamilyLaw.com.

Disclaimers:

The content provided in this podcast episode is for informational purposes only. It does not constitute the establishment of an attorney-client relationship. Louis’ primary office is located in Louisville, KY. This is an advertisement.