AI firm Anthropic has admitted its Claude AI fabricated a legal citation subsequently used by its lawyers, Latham & Watkins, in an ongoing copyright lawsuit. The company formally apologized for what it described as “an embarrassing and unintentional mistake,” after music publishers brought the erroneous citation to the court’s attention.
This incident throws a harsh spotlight on the persistent reliability issues plaguing artificial intelligence in high-stakes professional environments. Anthropic’s court filing detailed how its Claude.ai model provided a citation with what the company termed “an inaccurate title and inaccurate authors,” an error that regrettably slipped through manual review processes. The admission adds to a growing list of AI-generated misinformation cases, prompting serious questions about the current readiness of such technology for critical applications and underscoring the indispensable nature of meticulous human oversight, particularly as investment in legal AI technologies continues to accelerate.
Ivana Dukanovic, an associate at Latham & Watkins, stated in a court declaration that after her legal team identified a relevant academic article through a Google search to potentially bolster expert testimony, she tasked Claude.ai with generating a properly formatted legal citation using the direct link to the correct article.
However, the AI returned a citation that, while including the correct publication title, year, and link, featured an incorrect article title and erroneous authors. This critical mistake, along with other AI-introduced wording errors in footnotes, was not caught during the law firm’s manual citation check. This all transpired within Anthropic’s legal defense against music publishers who initiated a lawsuit in October 2023, alleging that the Claude AI was unlawfully trained using copyrighted song lyrics.
Court Scrutiny And Industry Precedents
Before Anthropic’s formal admission, U.S. Magistrate Judge Susan van Keulen had ordered the company to respond to the allegations. Judge van Keulen described the potential use of a fabricated citation as “a very serious and grave issue,” as Music Business Worldwide reports. During that hearing, Matt Oppenheim, attorney for the music publishers, revealed he had contacted one of the purported authors and the journal, confirming no such article existed, and suggested, “we do believe it is likely that Ms. Chen used Anthropic’s AI tool Claude to develop her argument and authority to support it.”
In defense, Anthropic’s attorney, Sy Damle of Latham & Watkins, contended it was likely a “a mis-citation” rather than an outright fabrication. Judge van Keulen pointedly remarked on the “world of difference between a missed citation and a hallucination generated by AI.” The music publishers went as far as to urge the judge to sanction the Latham & Watkins attorneys for the oversight, according to Law360.
This situation is not an isolated incident within the legal tech sphere. A recent report from The Verge highlighted another recent instance where a California judge criticized law firms for submitting “bogus AI-generated research.” Furthermore, a Sorbara Law article discusses a 2025 Ontario case, Ko v. Li, where Justice Myers emphasized that it is the lawyer’s duty “to use technology, conduct research, and prepare court documents competently,” as a warning against errors stemming from AI use.
While Anthropic had previously reached a settlement in January 2025 with music publishers concerning Claude generating lyrics, the fundamental dispute over the legality of training AI on copyrighted material persists. In March 2025, Anthropic had secured a procedural victory when a judge denied an injunction request by the publishers.
AI Hallucination: A Persistent Challenge
The phenomenon of AI models generating confident-sounding but entirely false or misleading information, widely known as ‘hallucination’, continues to be a significant hurdle for the artificial intelligence industry. Meta’s AI, for example, faced public backlash in July 2024 for incorrectly denying a major, widely reported news event.
Joel Kaplan, Meta’s global head of policy, candidly referred to hallucinations as an “industry-wide issue.” and acknowledged that all generative AI systems can, and do, produce inaccurate outputs. Worryingly, even newer and supposedly more advanced models are not immune; OpenAI’s o3 and o4-mini models, released around April, reportedly exhibit higher hallucination rates on some benchmarks than their predecessors. An OpenAI spokesperson conveyedthat addressing these fabrications remains an active and ongoing area of research for the company.
Independent research, including work by Transluce AI, has suggested that certain AI training techniques, such as reinforcement learning, might inadvertently amplify these hallucination issues in advanced models, a perspective shared by researcher Neil Chowdhury in discussion with TechCrunch. Other technology companies have also had to publicly address AI blunders.
For instance, Cursor AI’s customer support chatbot invented a fictitious company policy in mid-April 2025, which led to significant user backlash and a swift apology from its co-founder. He admitted their “front-line AI support bot” was responsible for the incident. Commenting on the broader implications of such incidents, former Google chief decision scientist Cassie Kozyrkov stated, “this mess could have been avoided if leaders understood that (1) AI makes mistakes, (2) AI can’t take responsibility for those mistakes (so it falls on you), and (3) users hate being tricked by a machine posing as a human.”
Navigating AI In Legal And Other Critical Fields
The legal profession, among others, is actively grappling with the complexities and responsibilities of integrating AI tools into its workflows. A study presented at the CHI 2025 conference revealed a curious finding: individuals might show a preference for AI-generated legal advice if they are unaware of its origin, despite the known unreliability of current AI systems.
This tendency raises significant concerns about the potential for users to act on flawed or entirely incorrect information. In response to these challenges, the AI research community is actively pursuing solutions. One such example is the SAFE (Search-Augmented Factuality Evaluator) system, developed by researchers from Google DeepMind and Stanford University, which is designed to enhance the truthfulness of AI chatbot responses by cross-referencing generated facts with information from Google Search.
Anthropic has assured that it is implementing new internal procedures to prevent similar citation errors from occurring in the future. This incident, however, serves as a potent, industry-wide caution: rigorous and diligent human verification remains absolutely critical as artificial intelligence tools become increasingly integrated into professional and critical workflows.
The path forward will undoubtedly require the development of more dependable AI systems, coupled with clear, robust usage guidelines and potentially new regulatory frameworks, such as the transparency mandates seen in the EU AI Act, to foster public trust and ensure accountability in the rapidly evolving age of generative AI.
