Law Firms Don’t Know Why They’re Buying AI – Artificial Lawyer

In this week’s Law Punx blast, Horace Wu says that law firms don’t really know why they’re buying AI tools.

Wu, who runs legal tech company Syntheia, argues that many firms are wasting money on AI tools without a clear understanding of their return on investment (ROI). He emphasises the importance of training staff to leverage AI effectively rather than relying on external platforms. The Law Punx episode also touches on the balance between convenience and the actual value provided by these technologies, questioning whether firms are truly gaining a competitive edge or simply keeping up with the market.

You can watch the seven-minute blast here, please press Play, or you can watch this and browse through over 170 other legal tech videos on the AL TV Channel here.

This episode and four other Law Punx podcasts are on Spotify, which also features Electra Japonas, Jake Jones, Todd Smithline, and Richard Mabey.

The Spotify Law Punx site with all episodes is here – where you can listen/watch all of the episodes in one place.

And the same, but just an audio version, is available on Apple Podcasts.

Several more Law Punx blasts have been recorded are now in the studio, including Oz Benamram and Jerry Levine, among others. Drop AL a line if you’ve got something to say!

Takeaways:

Law firms are overspending on AI without clear ROI.

Top-tier lawyers may outperform AI platforms in expertise.

Training internal staff is crucial for effective AI use.

Convenience is a valid reason for adopting AI tools.

Accuracy does not always equate to usefulness in AI outputs.

Many firms buy AI tools to avoid being left behind.

The value of AI tools should be critically assessed.

Convenience does not guarantee quality in legal services.

And here’s the AI transcript

Richard Tromans (00:10.225)

Hi Horace, good to have you on the show. Before I hand over to you, let’s just preface that idea. As Horace explained it to me, it is basically that law firms are wasting a lot of money, particularly on AI.

Horace Wu (00:40.952)

Well Richard, thank you for having me and listeners, thank you for giving me these couple of minutes to rant. And I’m going to start by telling you something you should already know, which is law firms are wasting a huge amount of money right now on genAI. And everyone’s kind of going out there claiming they’ve got return on investment from investing in genAI products and whatever they’re doing. And I would say that is absolute BS for a number of reasons. One.

The calculation of ROI is a financial metric. It is all about money gained, money saved or money retained. And a lot of law firms are claiming ROI as this fuzzy concept of, it makes people feel good. It’s introducing efficiencies. It’s saving us time. Unless those things are mapped against dollars, that’s not ROI. That’s something else. So let’s talk about what this something else is and whether it’s actually worth the money that you’re spending on this? And I think the answer is also no to that, by the way. So right now, there’s a lot of people who are going out there buying products like Harvey and Legora and other platforms. And I think they’re good platforms for what they are. But is it really giving people that much of an uplift from subscribing for a much cheaper direct subscription to OpenAI or Claude or Gemini? Questionable, questionable. So let’s kind of assume that these platforms do give you some additional value. Things like convenience, it’s got pre-made workflows, things like it’s got inbuilt domain expertise because the smart folks at Harvey and Legora and the other companies are all building in their domain expertise. So you can just click a button and get a response. Well, if you’re a top tier firm, do you think your lawyers are smarter than the folks at Harvey and Legora? Probably.

Probably, even if you’re not a top tier firm, are your folks gonna know your subject matter better than Harvey and the Legora and the other folks? Probably. And it comes down to if you know what you’re prompting into these models, you can get these models to do what you want rather than what someone else has pre-programmed them to do, using whatever it is they’re doing. So let’s strip that away, right? So if you know what you’re doing, then that’s gonna give you a better result using a raw model like.

Horace Wu (03:04.48)

OpenAI or Claude. So are you investing in training your people, in teaching them how to play with these machines? And I have a story. have a story. And the story is actually this morning, my brother messaged me and he’s looking to buy a property. And he’s like, hey, what does this contract say? What does this contract for sale say? And I’m like, I don’t know. I haven’t done conveyancing for like 15, 20 years.

And he said, why don’t I just put it into GPT? And I said, you can, and it can probably tell you the answer, because these are public documents, and they’ll definitely be in the training set. The question is, do you know the right question to ask? And that comes back down to, do you trust your people, your law firm people, your in-house people to know the right questions to ask, or do you trust some external vendor to know the right questions to ask? And if the answer is you trust your internal people, then you’ve got to ask,

Am I spending the money in the right way? Does it make me feel better to outsource my thinking and outsource my planning and outsource my prompting to someone else to do it? And the answer may be yes, but you have to consciously decide that because otherwise you’re wasting your money. And I’m going to make two more points about wasting money. The first is there’s a difference between 80 % accurate versus 80 % useful. And I think a lot of times people are conflating the two.

people coming out, they’re going, oh, we’ve now got these machines that are 95 % accurate. It’s like, okay, it’s 95 % accurate, but you still have to check every single word of the output. Is that 95 % useful in that case? questionable, right? So all of these kinds of things are making you think about, is it really giving us actual return on investment or is it making us feel better? And if it’s making you feel better, then buy it on the basis it’s making you feel better.

There’s an example, and this is the final story I have in mind, and I’m going to maybe talk some more, but maybe wrap it up. There’s a famous example right now of Freshfields going out there partnering with Google to build stuff. And it’s a very unusual strategy because no, not no other firms, very few other firms are going out there building their own things. Most people are buying Harvey and Legora. And I would argue that the reason you buy Harvey, Legora and platforms is because

Horace Wu (05:25.984)

you are having FOMO. You don’t want to be left behind. And that’s a key phrase here. If your motivation is, don’t want to be left behind, then buying those platforms, absolutely the right decision. Are you going to pull ahead? Are you going to have a competitive edge from buying those platforms? Probably not. Probably not. So if your motivation for spending your money is you want to feel good, you don’t want to be left behind,

then I would say, yes, go and buy those platforms. It’s a ready-made solution that you’re going to spend millions and millions of dollars on. I wish you’d give me that money, but no, no, no, go pay those people and get what you want. But if what you’re trying to do is pull ahead, take advantage of the market, take advantage of this technology, which is so magical, then are those the right decisions to be making? Should you be instead spending your money on training your people, on getting advantages from taking their knowledge and their expertise and

putting it into a platform that will give you so much more flexibility. That’s the question. That’s why I say, Richard, people are wasting their money right now. Because if it is true that most people argue that we are investing in Gen.AI to have a competitive edge, then the investment in the platforms, I would argue, is not giving you a competitive edge. It’s letting you keep up with the Joneses. And if that’s your real motivation for buying these platforms, congratulations, you’ve achieved your goal.

Richard Tromans (06:55.013)

Fantastic. Interesting, interesting, controversial, good stuff. Quick questions, we’re almost out of time, but one very, very quick question. Isn’t, might say convenience, yeah, that’s probably the simplest way of putting it, isn’t convenience something that we all like, right? You know, we all like using Uber, we all like using delivery services and so on. Is there anything really bad about Big Law firms with many many lawyers just saying: hey look convenience, let’s just bring it all together one product, it’s very simple

Horace Wu (07:29.322)

I would argue that is a great reason to buy these platforms. If convenience is the motivation factor, I can go down to 7-Eleven at the corner and buy some food. It’s super convenient. Is it tasty? Is it gourmet? Is it the type of food that clients pay $1000 an hour to get? No, it is not. But it’s convenient and that is a great reason to buy 7-Eleven.

Richard Tromans (07:49.307)

All right, to be continued, but thanks Horace, awesome stuff. Thank you very much.

Horace Wu (07:58.03)

Thank you, Richard.

—

Legal Innovators Conferences in London and New York – November ’25

If you’d like to stay ahead of the legal AI curve then come along to Legal Innovators New York, Nov 19 + 20 and also, Legal Innovators UK – Nov 4 + 5 + 6, where the brightest minds will be sharing their insights on where we are now and where we are heading. 

Legal Innovators UK arrives first, with: Law Firm Day on Nov 4th, then Inhouse Day, on the 5th, and then our new Litigation Day on the 6th. 

Both events, as always, are organised by the awesome Cosmonauts team! 

Please get in contact with them if you’d like to take part.