There has always been some level of risk involved when it comes to gen AI, from hallucinations to biased output based on training data.
However, a new report published earlier this month found that certain large language models (LLM) are less secure than expected, giving legal professionals a reality check for potential cybersecurity breaches and unwanted data retention.
After analyzing 10 popular LLM providers including Open AI, DeepSeek, among others, Cybernews researchers, through its Business Digital Index, gave low security risk ratings to half of the providers it considered. LLM providers Open AI, 01.AI, Inflection AI, EleutherAI and DeepSeek received medium to critical risk ratings.
All of the LLM providers analyzed had varying degrees of vulnerable encryption and half of the providers also recorded data breaches. Perplexity AI and EleutherAI in particular reported credential leaks.
Although law firms have been ramping up their cybersecurity protocols, SocialProof Security CEO Rachel Tobac told Legaltech News that an LLM’s lack of security means that LLMs could retain user data that ends up appearing in breaches.
“A lot of these AI tools end up having breaches, for instance, within 24 hours of Gemini’s chatbot LLM release, people who were searching on Gemini were finding their private search results showing up in Google,” she explained. “If you wouldn’t post it on your Facebook or your Instagram, I would not enter those exact details into the AI tool … anything that you have that’s proprietary, sensitive, anything that you really need to keep private, I would not enter into those tools.”
Tobac added that regardless of proprietary agreements between law firms, legal tech vendors and LLM providers that limit how data is used or accessed, breaches of sensitive information can still occur.
“Just because somebody has a contract with somebody and promises to do something a certain way, it doesn’t mean that things won’t go wrong. In fact, the majority of the time, when somebody has a breach, they have no idea that they’re breached at all until someone brings it to their attention … having a contract does not eliminate the risk of a breach,” she said.
On an organizational level, when a law firm uses a suite of products, it’s possible that users may not know that third party LLMs are powering the software they could be putting their data into, and may be at risk if breaches of these models occur.
“If you use a Google suite and you have access to Gmail—you have access to Gemini … you may have no idea of how that information is being used,” Silvino Edward Diaz, the chair of the entertainment law group at EPGD Business. “Whether it’s one of these large language models that they’re using for either research or data scrubbing for summarization, for e-discovery, for document production and evidence, and also the platforms that they use … there needs to be a higher level of understanding.”
Diaz added that it’s of the utmost importance for legal teams to vet their technology vendors and narrow down who within the organization has access to sensitive data in the event of a cybersecurity incident.
“Even if you have those contracts … you have to make sure that you have a specific protocol within your organization that says, all right, who has access to this data? What is their title? When can they have access through which devices are they going to have access to this? When can they access this? When do they have to notify several key members,” he said.