Lake Merritt is a new, open source, custom AI evaluation system that helps legal, risk, and product leaders ‘define what good looks like’, or in other words: gauge whether an AI tool works as intended. It’s been developed by Dazza Greenwood, the well-known legal tech expert.
The initial version of Lake Merritt is an Alpha release and is aimed at law firms, inhouse counsel, and regulated industries ‘where risk, compliance, and trust cannot be left to generic benchmarks, [because] evaluation is not just a technical task – it is a strategic function’.
US-based Greenwood told Artificial Lawyer: ‘Lake Merritt is an open-source, Alpha-stage platform that puts custom AI evaluation – especially for legal use cases – directly into the hands of lawyers and business leaders.
‘It is designed to answer the most important AI governance question: how do we know this system is working the way we need it to? Instead of leaving evaluation to engineers alone, Lake Merritt enables domain experts to define what ‘good’ looks like, create ‘golden datasets’, and test AI behaviour against real business and legal priorities.’
He added that the platform has already been used in client and collaborator settings, ‘supporting evaluations for product release, fairness benchmarks, complex multi-agent systems, and even identifying subtle breaches of the fiduciary duty of loyalty’.
However, he stressed that this Alpha version is very much a ‘research preview’ and will see: new workflows, eval packs, and extra features over the coming weeks and months.
The main hub for information, tutorials, software links, and updates is here and that’s also where you can provide feedback, sign up for AMA-style Zoom sessions, and follow the ongoing development of Lake Merritt.
Key entry points are intentionally simple, Greenwood added, and you can ‘start with CSV-based evaluations, scale into more complex multi-step agent testing, or use the ‘Hold My Beer’ workflow to rapidly bootstrap evaluation datasets from plain-language requirements. The ‘Eval Pack’ architecture makes the process more transparent, version-controlled, and designed to be iterated as your needs grow,’ he concluded.
And here is a short video explaining some more.
First, do we need additional eval systems? Yes, and especially so in the legal field where there are just a handful. Plus, this is not just ‘another eval’, it’s operating from a different perspective, i.e. it is asking whether an AI tool you are using is doing what you expected from it.
It’s also designed to allow people who are not engineering experts – but do know their legal niche – to explore and understand the performance of the legal AI tools they’re onboarding.
This may help legal innovation teams to run better POCs and ‘bake offs’, as well as improve overall insights into how existing AI tools within a firm or inhouse team are performing vs expectations.
More info here.
—
And, as Dazza mentioned above, please get in contact with him via the links above if you’d like to give feedback or know more.
—
Legal Innovators Conferences in London and New York – November ’25
If you’d like to stay ahead of the legal AI curve then come along to Legal Innovators New York, Nov 19 + 20 and also, Legal Innovators UK – Nov 4 + 5 + 6, where the brightest minds will be sharing their insights on where we are now and where we are heading.
Legal Innovators UK arrives first, with: Law Firm Day on Nov 4th, then Inhouse Day, on the 5th, and then our new Litigation Day on the 6th.
Both events, as always, are organised by the awesome Cosmonauts team!
Please get in contact with them if you’d like to take part.
Discover more from Artificial Lawyer
Subscribe to get the latest posts sent to your email.