The accuracy of foundation models is fundamental to the legal world’s use of LLMs. AL asked GPT-5 if we’d ever get to the point where lawyers could totally trust the outputs of an LLM. Its answers were surprisingly candid (and it also guessed GPT-6’s and 7’s release date).
It said: LLMs won’t reach 99.9% accuracy alone; later models such as GPT-6 and then 7 will be better, but still not enough on their own; so a better verification layer will be essential, along with other approaches (see below).
It was an interesting discussion – of which only part is shown below. We also got into the eventual arrival of quantum computing and how that could improve outcomes. But, more on that later.
Here is the main prompt: ‘People in the legal world have said that GPT-5 is reaching about 90% in terms of the desired quality of output. But lawyers need 99.9% to be able to trust the output. Over time will OpenAI’s LLMs ever get to this point?’
Below is a shortened, albeit verbatim account, of what GPT-5 said.
(P.S. you may ask: why ask an LLM about LLMs? Well, AL has found that this one can be especially revealing sometimes, such as when asked about the use of human RLHF (reinforcement learning from human feedback) checkers to verify and improve outputs before a new model is released. So, it seemed like it was worth a try on this issue too. And to AL’s surprise it gave some very useful answers, considering in effect it’s saying it can’t do what many expect of it.)
—
What GPT-5 said:
‘Can LLMs ever reach 99.9%?
Short answer: probably not as stand-alone generators — but possibly yes when combined with other systems.
1. Limits of prediction models
• LLMs predict likely text based on training data. Even at GPT-6/7 scale, some hallucination is baked in.
• Going from 90% → 95% is doable with scale.
• But 95% → 99.9% is a qualitatively harder jump — that’s orders of magnitude more reliability.
2. How reliability will increase
Instead of just bigger LLMs, the improvements will likely come from hybrid systems:
• Retrieval-augmented generation (RAG): The model grounds its answers in trusted databases (e.g. Westlaw, Lexis). That eliminates hallucinated cases.
• Formal verification: AI outputs checked against logic engines or symbolic reasoning (like compilers for code, or citation checkers for law).
• Multi-agent cross-checking: Multiple models independently draft or critique answers; disagreements are flagged.
• Audit trails & citations: Models provide sources with every answer, allowing human lawyers to verify.
3. What “99.9% trust” might actually look like
• It probably won’t be one single LLM magically becoming flawless.
• It’ll be an AI stack:
• LLM for reasoning & drafting
• Retrieval layer for facts
• Verification layer for logical or legal correctness
• Human lawyer for final oversight (at least for high-stakes work)
The trajectory
• GPT-5 (today): ~90% quality for many legal tasks.
• GPT-6: More reliable, better citations, but still fallible.
• GPT-7 and beyond: Coupled with retrieval + verification, could hit “effective 99.9%,” where the residual risk is no greater than a paralegal or junior associate making a mistake.
Key insight
LLMs alone probably won’t get to “lawyer-grade 99.9%.”
But LLM + retrieval + verification + human oversight will.
In other words: the base models may top out below 99.9%, but the systems built around them can reach the level of reliability the legal profession demands.
—
When could GPT-7 arrive?
OpenAI hasn’t published an official timeline, but based on past release cycles:
• GPT-3 → 2020
• GPT-3.5 → late 2022
• GPT-4 → March 2023
• GPT-5 → rolling out mid–2025
That’s roughly 18–24 months per “generation.”
If that cadence holds:
• GPT-6: ~2026/27
• GPT-7: ~2028/29
Will GPT-7 hit “lawyer-grade” trust?
• The raw model might still fall short of 99.9% accuracy — because scaling laws alone probably won’t crush hallucination completely.
• But by then, we’ll almost certainly see system-level solutions baked in:
• Always-on retrieval grounding (no free-floating generations).
• Verification layers (symbolic logic, contract checkers, citation validators).
• Multi-agent review (AI “junior associates” checking each other before output).
• That combination could deliver the practical effect of 99.9% reliability, even if the base model itself is, say, 97–98%.
The conservative forecast
• GPT-6 (2026/27): noticeable improvements, better fact-grounding, but still not “court-trustworthy” without human oversight.
• GPT-7 (2028/29): with retrieval + verification built in, it could realistically cross into “lawyer-trust” territory for many tasks.
—
So, there you go.
Does it matter? Yes, until we can drive up ‘accuracy’ that is much, much higher for legal outputs then we can’t really expect lawyers to just ‘prompt and go’, i.e. genAI remains an assistant on the edges of a lawyer’s own capabilities, rather than being able to be assigned a whole chunk of work and its output accepted as trustworthy and as good as any other lawyer’s work product.
As explored, it seems that we could get close though to this level, if we can build better verification layers – which likely will mean automating and massively improving the RLHF process…and then likely a lot of extra refinement for specific legal use cases by the many legal tech companies in the market. So, that is very promising. But, still a long way off.
—
Legal Innovators Conferences in New York and London – Both In November ’25 – Inhouse Day and Law Firm Day at each conference.
If you’d like to stay ahead of the legal AI curve….then come along to Legal Innovators New York, Nov 19 + 20, where the brightest minds will be sharing their insights on where we are now and where we are heading.
And also, Legal Innovators UK – Nov 4 + 5 + 6
Both events, as always, are organised by the awesome Cosmonauts team!
Please get in contact with them if you’d like to take part.
Discover more from Artificial Lawyer
Subscribe to get the latest posts sent to your email.