College students who have reportedly grown too dependent on ChatGPT are starting to face consequences after graduating and joining the workforce for placing too much trust in chatbots.
Last month, a recent law school graduate lost his job after using ChatGPT to help draft a court filing that ended up being riddled with errors.
The consequences arrived after a court in Utah ordered sanctions after the filing included the first fake citation ever discovered in the state hallucinated by artificial intelligence.
Also problematic, the Utah court found that the filing included “multiple” mis-cited cases, in addition to “at least one case that does not appear to exist in any legal database (and could only be found in ChatGPT.”
Douglas Durbano, a lawyer involved in the filing, and Richard Bednar, the attorney who signed and submitted the filing, should have verified the accuracy before any court time was wasted assessing the fake citation, Judge Mark Kouris wrote in his opinion.
“We emphasize that every attorney has an ongoing duty to review and ensure the accuracy of their court filings,” Kouris wrote, noting that the lawyers “fell short of their gatekeeping responsibilities as members of the Utah State Bar when they submitted a petition that contained fake precedent generated by ChatGPT.”
The fake citation may have been easily caught if a proper review process was in place. When Ars prompted ChatGPT to summarize the fake case, “Royer v. Nelson, 2007 UT App 74, 156 P.3d 789,” the chatbot provided no details other than claiming that “this case involves a dispute between two individuals, Royer and Nelson, in the Utah Court of Appeals,” which raises red flags.