Trial by Algorithm: Helping Lawyers Navigate the AI Revolution

Katya Valasek:

We're joined today by Joe Stephens. Your career post law school has been non-linear. On that theme, we'll address your career history a bit later, but to start, I want to talk about your role as a trial consultant. You're a lawyer, your clients are lawyers, and you're not a lawyer for their clients. At what point does another lawyer typically come to you for help?

Joe Stephens:

They come to me when they're trying to marry the thing that they know best, which is their practice, whatever that niche is, and this new technology that exists. Because around their practice, around every lawyer's practice right now, is this sort of addition of AI. You can't open a journal, you can't go to a conference without reading about it.

And so people, unlike any other technology, probably in their practice career, no matter the depth of it, this is upending it. This is affecting it. And I think that people are realizing that if they're not using AI to some degree, that they might be losing business to someone who is. So they come to me when it comes time to sort of ask those questions about, okay, how can I use this to affect and better my own practice?

Katya Valasek:

What are typically some of the issues that may be going on in a practice that makes them start thinking, “I should explore AI as a way to help?”

Joe Stephens:

AI is a really interesting reflection, I think, sort of of our society at large, in the sense that people are now conscious of the things that take them the most time to do, and feeling like AI could help them solve that problem. So, for example, a really specific use case would be a lawyer who is preparing for trial and has a mountain of depositions to sort of understand and process, and not just search through but really understand. You know, where their witness is contradicting each other, like help me develop different trial themes or jury selection questions or drafting motions in limine, whatever it might be. How can AI affect this part of my practice?

How can it clean up and tidy up sort of this busyness here and allow me to do the thing that I'm best equipped to do, which is: have the information and decide what to do with it. So that's when people come to me and ask me if I can sort of like strategize with them on the time saving front when it comes to things like deposition analysis or trial prep off that deposition analysis.

Katya Valasek:

So when people think of AI, and this is myself included, I'm mostly thinking of generative AI, like chatGPT. Is that what this AI is in the litigation space? Are there different levels of AI?

Joe Stephens:

Yeah, there's different kinds for sure. I think that most people think the same way about it that you do, Katya. And I think generative AI is what is most disruptive right now. It's interesting though because people get hesitant to use it, even though if you think about the job of a trial lawyer or litigator in general, it really revolves around asking questions. It's taking information and figuring out how to ask opposing counsel questions in a contested setting, obviously a witness. And yet when it comes to AI usage, that same principle can actually be a little jarring to people. Like how do I get the information out of it that I want? And it revolves around the quality of questions that you ask it.

Katya Valasek:

How do you decide to take a new client? Do you evaluate what their practice is? Or do you believe that anyone would benefit from AI in their practice?

Joe Stephens:

There's a couple of answers to that, right? I mean, as a public defender, I have no discretion when it comes to the clients that I take. They are appointed to my office based on income levels and whether or not they fall below a certain poverty threshold. So I don't get to pick it on that front. When it comes to the AI work that I do, there is more discretion.

But I do believe that what you said is right.I think it's sort of subject matter agnostic. I think that anyone can and should benefit from the AI usage itself. It's there, it's transformative, and it's kind of magical.

I think once you vet it security wise, because there's obviously a lot of ethical obligations that we have as lawyers, and you don't want to sort of disrupt those in any way, shape, or form. But once you sort of vet the products for how they can do what they do in security wise, I think you should be using it. It's just this magical tool that allows you to do so much more, so much more thoroughly at a much more rapid pace.

Katya Valasek:

How did you grapple with that security piece when you were dreaming of and first stepping into the space as a consultant?

Joe Stephens:

When people talk about AI security, I think there's two sides to it. One is ensuring that the product that you're using, first and foremost, is doing the thing that it purports to do. And so you've got to be able to sort of back end check to make sure that if I'm putting any information in here, I need to know what you're doing with that information.

And I think AI companies are much better now about being transparent around the security protocol that they offer because they have to be, especially if you're engaging with lawyers. You have these already sort of preset ethical obligations to their clients.

On the back end though, there's another form of security that is obviously much more user focused. You have to be able to understand and verify that what it's telling you is correct. This is where we see people get into trouble, right? Like these are the headlines about people putting things in pleadings, cases that don't exist, cases that don't stand for what they say it's actually standing for. So making it so that it's so user friendly on the back end, so that I can check to make sure that it's telling me the right thing. Those are the two things. It does those things, then I think you're golden.

Katya Valasek:

So a lawyer comes to you. They're looking to integrate AI into their practice. What is the very first thing you do for that lawyer?

Joe Stephens:

I will say, what is the part of your practice area that takes the most time, is sort of the most repetitive? And then we'll start a conversation from that point and try and discuss with them AI solutions. And now find a specialized tool that does it brilliantly.

A really common example is I have a mountain of data. How do I understand this data? How do I understand what lives in these transcripts? How do I use the information within these transcripts, these depositions that I've taken to either prepare for other depositions, figure out who else to depose, figure out what else to ask for on the discovery front, or just really get my trial preparation nicely buttoned up. So that's where I see it the most. But it starts again with the questions, like where are you needing the most assistance in your practice? Because there's an AI tool out there for you to help with that.

Katya Valasek:

You're a lawyer. Before you went to law school, you were involved in politics. How did this interest and this understanding of how technology, specifically how AI can help a lawyer come into being part of your wheelhouse and part of your expertise?

Joe Stephens:

I've always seen, though I've wrestled with this, I've always seen my law degree as a tremendous privilege. And I mean that. You walk around with something that a lot of people don't have. And the courthouses around this country remain an area where you can stand up and make a difference in someone's life. That matters to me a lot. I've also always come at this from more of a social justice front. You're entrusted with someone's story. And I don't know if there is a more beautiful thing in the world than that.

Those two things together mean that, in my mind, you have a sort of obligation as a lawyer to stay up to speed on the ways that you could better your practice. And when you talk to somebody who's been practicing for 50 years or talk to somebody who's been practicing for 10 years, this is the recent entrant into the field, this AI technology that allows you to do something so much better with way fewer resources. And once you know that this thing is out there, I sort of think that if competent representation is the name of the game and every single state bar is telling you that obviously that's the obligation that we have as lawyers, and ideally more than that, then AI is necessarily a part of it.

Katya Valasek:

I love what you just said, because essentially then what you're doing when you're consulting is you're helping these lawyers become better at their practice. As you're working with them through this, are you in the courtroom with them while they're in trial or just helping on the front end, helping with reflection along the way? Once you get someone set up, how involved are you in the day to day?

Joe Stephens:

It can vary. As we all know, sort of like 100% of cases, civil or criminal, negotiate at some point. But a very small percentage actually wind up in trial. The willingness to go to trial and be exceptional in trial are the most powerful levers we have as lawyers in the litigation world. So very often I'm prepping as if we are going to try a case.

One of the most recent cases I worked on, I came in with a very solid trial setting after years of negotiation and years of this case pending. But because of the work that we did, it settled. I would have been in the courtroom during trial. I would have been there assisting, taking the information that was given and then consulting day after day after day.

But because these cases haven't actually been tried, I'm just there on the prep front. And sometimes I get sort of parachuted in 100 days before trial and it becomes go time. All right. Now we're ready to prep witnesses and we're ready to sort of get our filings underway and we're ready to really start drafting and putting together a trial notebook that allows us to be exceptional in that.

So, I've never actually sat in the courtroom. I would love to because trial work is some of my favorite work, but that's the thing about using this technology. It gives you the levers that allow for really powerful negotiation prior to a case going to trial.

Katya Valasek:

In that example you just gave, you said that because of the technology and because of your consulting, the case settled instead of going to trial. How did the technology give your clients the edge on behalf of their clients to push it over the edge into settlement instead of actually going into trial?

Joe Stephens:

So this case was a police brutality case. This was a case of a man who spent nearly 30 years in prison for a crime that he didn't commit. He had an innocence certificate. There was no contesting his innocence. The issue was a lawsuit against the City of Chicago. This is a public case that has settled. So everything we talk about is obviously very okay here.

When I got brought into this case, they were getting ready to try it because there had been no meaningful progress when it comes to negotiation and settlement. Coupled with the willingness to go to trial, excellence in trial matters tremendously. And excellence in trial comes from one of two things. One is your history of being in trial and people know how good you are in the courtroom. The second is you at every single corner are the most prepared person in that courtroom. You know all the facts. You have answers to all the questions. There's a rock-solid confidence in your position. That was the thing that AI did.

I would think that we took in that case the amount of documents, the amount of filings, the amount of depositions. This is a case that had been up and down from the highest court in Illinois. We took all of the information that we needed and basically ran it through AI to do everything from spend a day running our client through an anticipated cross-examination, finding other inconsistencies, drafting really more comprehensive motions. We came into that in the span of about 100 days just looking like, because we were, exceptionally on top of what was going on in this case. That meant something.

I mean, that brought, in this case, the City of Chicago back to the table and allowed the lawyers who were working on that case to negotiate it because they just had every single aspect of the case covered.

Now, you can do that without AI and people have done that for the entirety of the history of being lawyers. I was able to work with the trial team here and do it in the span of two weekends. The amount of cost savings because of the settlement—the amount of money that we did not spend on traditional lawyer services and time—meant that for this man, a bigger portion of the settlement went into his pocket. And I just think that that's right.

Katya Valasek:

You have mentioned a few times now that AI can really assist in trial prep, going through large quantities of documents and summarizing and helping get a basic understanding. Can you think of another example of where AI can help that is maybe not as obvious or readily apparent to someone who isn't doing the work themselves?

Joe Stephens:

It can really heavily assist in your motions practice. Anytime you're drafting anything, and I think the ability to write matters tremendously. I grew up in a house of—my dad was a journalist. So the economy of words and the way to deploy words was something that was in my blood from day one. But I do think that if you are a lawyer who needs to persuasively and effectively write, I think you should be having some sort of AI assistance, whether it's checking it, whether it's asking if what you're trying to argue or what you're trying to say is missing anything, if it could be a little bit tighter. I don't see why you wouldn't use that.

It kind of pains me to say that because of the pride that—I suppose I'm thinking of my dad here—that he has in just words and he spent an entire career without using it. But I do think that it's there and it makes your work product better, tighter and your time management that much more efficient.

Katya Valasek:

So I can think of many, many movies where there's lawyers involved and they're prepping for trial and you get this montage of them in a conference room at all hours of the day with boxes and pots of coffee and takeout. It sounds like AI is sort of making that not as necessary as it may have been in the past. Do you ever worry that you're hastening the demise of lawyers?

Joe Stephens:

First of all, the Hollywoodization of lawyering is amazing to me. There's some truth to that. I mean, the prep work is not glamorous and glorious, but your question at large is, I think, much more interesting. Is AI hastening the demise of the legal profession? The answer is no, definitely not in my lifetime. I think headcounts within law firms are going to change when it comes to skillsets that people are looking for. I think that people, law firms generally are going to want people who are more tech adept and tech fluent. I don't think that's going to fundamentally upend the number of lawyers in those firms.

I think it'll hold people more accountable. I hope it reduces and eliminates the sort of practitioners who might just be sort of coasting by a little bit. But I really, really, really think what it's actually going to do is allow people to have, at the end of the day, much better representation. And I firmly, firmly believe that it is the greatest assistant that you have if you use it correctly.

But that said, let's be practical, right? There is no judge that I know of. There is no courtroom. There's no judicial district. There's no state bar. There's no legislature. That’s looking at this technology and seeing it in an excited way of getting people out of this process. It's going to free you up to do the thing that is the highest and best use of your mind. That, to me, is where this is the power of this in the next decade.

Katya Valasek:

Do you think pro se defendants or people who are representing themselves should be using AI to help represent themselves? Or is having someone who went to law school and has the training to be a lawyer and think like a lawyer critical to the process? Does it matter that a lawyer is using it rather than a pro se defendant?

Joe Stephens:

I think there's space for both. One of the things, to me, this is going to do, and I was talking about this to a bunch of new hires in a public defender office yesterday. I can teach you the rules of evidence. I can teach you civil procedure. I could teach you criminal procedure. Those things are commodities. It's not easy. I'm not saying that you just open the book and you understand it and you're good. But those things, to me, are more quantifiable. It's black letter law. It's cases interpret that law. That takes a unique skill. Again, it's not easy.

I think what we're seeing with AI is it shifts the priorities to the softer skills. If you ask AI to write you an opening statement and you give it all of the right information, it will write you an opening statement. At some point, you have to stand in front of a jury and deliver that opening statement. And if that opening statement is technically brilliant and beautiful, but it's not delivered in your voice, that's going to come across as inauthentic. So I really, really believe that there's going to be a shift in focus to the softer skills.

That's one piece to the answer to your question. The other answer that I have to your question is that I think it is making people come into lawyer consultations throughout their case. I think it's giving them better questions to ask. And it used to be that you would walk in and you'd talk to a lawyer and you would just entrust that lawyer with everything strategy-wise, and they would be the expert and you would just deferentially listen to what they were going to say.

There's some goodness to that, right? I mean, that lawyer should be the expert. That lawyer should know how to guide you through the process. But I think that what I'm seeing are that clients are now using it and come almost better prepared with questions that I'm not expecting because they're asking AI those questions. I don't think that's necessarily a bad thing, but I do think that the lawyer-client relationship is changing a little bit as a result.

Katya Valasek:

To be hired as a consultant, you need to have a depth of experience that lends credibility to what you're advising. So how has AI helped you in your work as a public defender?

Joe Stephens:

The public defenders, again, are notoriously overworked. You can't pick your cases. And you're there to meet people at their darkest moment and solve a lot of problems. But I think one of the things that matters to most public defenders, they're very mission-driven people. They're there to serve the most vulnerable of the population in their minds. And I think there's a lot of truth to that.

But you only have a fixed amount of sort of brain capacity and time. And so when you have this ability, you have AI that slots in and allows you to do things much more efficiently, it's transformative. In every single case now, it doesn't matter what jurisdiction you're in. If there's a random traffic stop that leads to a charge, the mountain of digital discovery is tremendous. So now with AI, you can actually use it as a first pass through discovery. And it doesn't replace your actual human review of it. But it does allow you to sort of, okay, this is six hours of camera footage from dashcam of a police car. Maybe there's 10 seconds worth of discussion in that video that matter. AI will pinpoint those 10 seconds and allow you to go right to that point in time and let you know that, by the way, there was nothing else picked up audio-wise that meant anything over the remaining four hours. Those kind of things are absolute game changers when it comes to public defense.

Katya Valasek:

You are an AI super user. I am as far on the other end of the spectrum as you could imagine. How long ago did it dawn on you in the work that you were doing that there's a tool that can help me do this?

Joe Stephens:

First of all, I kind of want to acknowledge your skepticism of this. I think that's really fair. I don't think that people should just be blindly jumping into this. I think that there should be some vetting of the products. But I do feel like because I've done that and seen the benefit, I'm less skeptical.

So it was a few years ago. I built a public defender office in rural Texas. It covered 12 counties. It remains the largest rural public defender office in the state of Texas. The work that office does is incredible. Before that office existed, there was nothing going on in those counties. So to go from nothing to having this office serving this incredibly powerful role of indigent defense in the middle of nowhere in Texas, we were really looking for creative solutions. It was at that time that AI was kind of just coming of age. And we were fortunate enough to marry an AI solution to what I was just talking about, discovery review into our case management system. And when that happened, that discovery review for a criminal defense lawyer who is a public defender is the thing that probably takes the most amount of time. So when we were able to see a solution to that that was incredibly proficient and catch things that humans were not catching, my goodness, it was a game changer when it came to representation.

Katya Valasek:

So AI has obviously freed up some time in your life because on top of running the public defender's office, on top of your consultation, you also are involved with the clinic at Texas Tech. I know some universities have strict policies about use of AI, but I imagine that you are encouraging the clinical students to use the AI to their advantage. Was there any conflict with the school policy that you had to resolve to be able to advocate for this tool in the clinic?

Joe Stephens:

No, not at all. And it's a good question because I'm sure that that could come up. Texas Tech's law school is fantastic. And Texas Tech's law school is the only law school in the country that has a public defender office inside it. So the privilege of working there and running the public defender office in the clinic, doing the clinic work is one that I think about every day. It's a joy.

The conflict is not a written conflict. The conflict is really a fascinating conflict. It's sitting in a faculty meeting and hearing how people who did not use AI or haven't used AI throughout a majority of their practice—and these are brilliant, brilliant minds—how they wrestle with a generation of people who instinctively use it. What does that mean to skill set development? What does that mean to critical thinking? So the conflict is more a philosophical one, I think. But Texas Tech's law school is one that totally encourages that level of innovation.

The other thing that's true is regardless of the conflict, these students are using it, right? They might not be using it on exams. They might be barred from using it in certain parts of their academic process. But this is a language that they're speaking. So how do you communicate as a professor to a group of people who are speaking a little bit of a different language? And I'm sure that there are different shades of that over the history of academia. But this is the most recent iteration of it. And I find that to be the most fascinating conflict here. Again, it's not one that's on paper. It's one that's more just sort of a philosophical one, I think.

Katya Valasek:

So it sounds like most of the clinical students are not like me. They are willing to embrace the technology. But can you think of any of your students where you saw the light bulb literally turn on in the moment when they realized what this technology could do to assist them?

Joe Stephens:

Absolutely. I had a student who was working on an appeal during the school year. He had an internship that prior summer. And I asked him to help with an appeal. An appeal begins with a digestion and understanding of what took place at trial. The way you understand what took place at trial is by reading the record.

Now, on one level, there's nothing that replaces that, right? Like you have to sort of read the record. But if you're engaging with the record for specific reasons, if you have issues, arguments that opposing counsel has made that you need to respond to, AI is magical at basically giving you some quality output when you put the entire record into it and you say, I am representing the appellant in this case. And here are the issues that have been raised or the issues that I need to respond to. Find every single instance during trial that might be relevant.

And he did this. This is exactly what he did. Immediately isolated all the information and actually found issues that opposing counsel had not raised and actually found deeper ways to support the argument than his bosses had suggested were present. And he did it in a fraction of the time. As someone who has worked on appeals and post-conviction writs, I sit here just wishing I had this at my disposal because then it frees me up to actually take the information and do something with it much more quickly.

Katya Valasek:

Do you think it's possible that any of your students that you're teaching about this technology, when they go out into the world and join a firm or join a nonprofit, wherever they end up in practice, do you think they may encounter skeptical employers who will be hesitant to let them use the technology that you've opened their eyes to?

Joe Stephens:

100%. And I think this is something that I'm seeing already. When I talk about consulting with other lawyers on cases that they're working on, sometimes the obstacle actually isn't the technology itself. It's internal. It's, you don't use that here because that's not the way that it's been done. Or a more positive spin on that is we have been doing things so well this way. Why would we change and bring this new technology in? So yeah, I think that they should anticipate it. I also think that it's okay. Like pushing back on it.

One of the greatest privileges of my career has been learning without AI. I have had to develop a skillset without AI that now I can use AI to augment. And I feel really grateful for that. It's sort of is like the time that I lived without having a cell phone. Like I really feel grateful that I know what it means to sit quietly by myself and not have this perpetual sort of engagement with technology.

That said, because I know that before having the skillset to actually deploy AI effectively is something that yields such tremendous results that I think that there is space in there for them to sort of advocate for it internally and demonstrate the results really, really, really easily. If someone comes to you now and says, I need you to review these five depositions and come up with a line of questioning for this witness or these witnesses, come back to me. I anticipate this will take you two weeks and you can come back and do it better just in the span of two hours. The results sort of speak for themselves. That said, there are always gonna be real, real kind of militant skeptics and that's just fine. The short answer there is I do believe that if people in the legal sector are not using it, then you're gonna be losing business to people who actually are using it.So I think that that's naturally gonna solve itself over time.

Katya Valasek:

You mentioned just now that you were grateful that you gained the skillset without the use of AI, without technology. But you're now teaching students who are in a completely different position. How has your experience of being in law school, starting your career without AI as a resource, impacted the way you approach your job in the clinic and interacting with students?

Joe Stephens:

I wanna underscore the first part of your question. That gratitude is real. I have a mountain of books at my house and I still think that there is nothing that will ever replace the physical experience of sitting with something and wrestling with something and not having the immediacy of answers. And I'm grateful for that.

Your question though is a really interesting one because it's one that I ask and I'm curious about with other people. When you're now teaching people who are used to having something tell them a really brilliant answer, what does that do to the development and evolution of their legal mind? Their mind in general, but definitely their legal mind. And I really think that what it comes back to is an absolute spotlight on the ability to deal with, handle, take on ,and excel in the soft skills.

So that's what we talk about with my students is, okay, great, you've got the information. Now what? There's a moment where you have to, as a litigator, where, and this is not changing anytime soon, where you have to take all of that and now do something with it. Go tell a story. I mean, that's what a jury trial is, is a storytelling emotions hearing. Your ability to get on your feet and sort of figure out who your audience is and say something convincingly. That is going to, I think, even be more and more and more pronounced.

And that's a really hard thing to teach. There's no AI tool out there that is going to let you authentically learn your own voice, in my opinion. So teaching that part of someone and trying to get them to uncover that, that's where I think the real magic lies.

Katya Valasek:

I know that when I was in law school, I was a flashcard girlie and I took my notes on my laptop, but I hand wrote all my outlines because, as you said, wrestling with the material, like physically writing my notes did a lot to help me understand, remember, and draw connections. Do you think there's a difference in how people using AI need to prepare for trial? Because you don't just need the answers. You need to internalize them and be ready to object or be ready to respond to something that opposing counsel says. If you're not reading case after case and underlining and highlighting and making notes, how then do you think people are going to internalize all that information they need to have at their fingertips?

Joe Stephens:

I don't know how they do. Preparation is still preparation. And as you know, I just think that authenticity is not quantifiable. And if you don't know something and you get on your feet and you start arguing something, generally speaking, you're going to be pretty quickly called out. Whether that's by a judge or opposing counsel, fine. The worst case scenario is the jury setting. I mean, juries are notoriously good at feeling whether it's theatrical in a genuine way or inauthentic. You do still, to your point, absolutely have to spend enough time with it to where it just exists in your being. And that's where the skill development comes.

The hard thing about that though, is that that takes time. So I don't really know where all of this goes, but I'll tell you that the thing that brings me the most joy is when I do see someone sitting with a book and a highlighter. As much as I love AI, like I do love it when I see someone doing that because it feels like I can watch the knowledge be a little bit more deeply and authentically absorbed in their body rather than just like an information overload that comes with asking AI a question.

Katya Valasek:

So to end, I'm going to ask a question that's on a little bit of a different track because, as I was preparing to talk to you, I noticed that in your free time, you are an amateur potter. You like to do pottery. And I was immediately struck by this dichotomy of you being at the forefront of current technology, helping people learn how to use it, incorporating it into their work. And on the other hand, in your free time, you sit down in this really ancient technology setting. Talk to me about why pottery is important to you in your free time.

Joe Stephens:

As a lawyer, you can be really close to the work and then not see something come of it for years. You can be working a case for so long and even then it might not go your way. And it can be taxing, draining, difficult. It can be beautiful. It can be hard. It's a whole range of emotions.

When you sit down in front of a wheel and five minutes before you sat down, there was nothing. And all of a sudden there's a bowl out of the most simple ingredients, I just don't know if there's something more beautiful. And then you take that bowl and you bring it home and you make a meal, put that meal that you made from nothing in that bowl that you made from, and you put it in front of someone. To me, it's the ultimate form of just beautiful hospitality and service. And it's rewarding. My job very often is, I mean, it's always rewarding, but it's very difficult. Pottery doesn't do that to you. Pottery is not going to speak back to you. Pottery is not going to tell you that your argument is dumb. Pottery is not going to tell you that your client's guilty. Pottery is going to actually generally come into being and then provide joy to someone else. And there's, I think, something just really glorious and wonderful about that.