While artificial intelligence is a promising tool for lawyers, a recent case in Colorado shows potential pitfalls when it comes to blindly trusting AI.
Colorado attorney Zachariah Crabill used ChatGPT to write a motion in a civil case earlier this year. The motion cited three cases to support Crabill’s position that were made up by the generative AI program, which the lawyer didn’t realize until it was too late.
Crabill worked for the Colorado Springs office of Baker Law Group when he filed a May 4 motion to set aside judgment on behalf of the firm’s client Christopher Zelaya Chavez. Zelaya Chavez retained the firm after a court entered a more than $30,000 judgment against him in November 2022 stemming from a dispute over payments and repossession of a car he sold to Donovan Gates last year.
Zelaya Chavez initially represented himself after Gates sued him, but later hired Baker Law Group. Crabill filed a motion to set aside default judgment May 4 but soon realized his mistake the morning he was set to appear in front of the court May 5, according to an affidavit filed May 11.
Crabill cited three fabricated Colorado Court of Appeals decisions that, according to the motion, would have been examples of when a court granted a motion for relief from judgment even though a pro se defendant failed to file a timely appeal since the failure to appeal on time was inadvertent, excusable and not the result of willful misconduct.
The fabricated cases were “Zerbi v. Colorado Division of Employment & Training, 940 P.2d 697 (Colo. App. 1997),” “In re Marriage of McQueen, 970 P.2d 1245 (Colo. App. 1998)” and “People v. Hall, 276 P.3d 456 (Colo. App. 2011).”
Judge Eric Bentley with the 4th Judicial District Court denied the motion and ordered Crabill to explain why he shouldn’t be sanctioned under Colorado Rules of Civil Procedure 11 and reported to the Office of Attorney Regulation Counsel for the “fictitious case citations.” In a later denial of a second motion to set aside judgment, Bentley underlined that his decision to deny the motion wasn’t based on the made-up cases.
ChatGPT is a text-generative artificial intelligence chatbot program developed by OpenAI that launched in December 2022. The program has been hailed for its ability to write sophisticated content based on user prompts. It has made waves in the legal industry after content written by GPT-4 passed the Uniform Bar Exam with a 297 score and two lawyers in New York made national headlines in May when they used ChatGPT to write a motion that cited made-up case law.
Since ChaptGPT was introduced, a number of more traditional legal research databases, like Casetext and LexisNexis, have used the program to create their own AI legal industry-specific programs.
According to an affidavit and notice of errata filed by Crabill, he was “overwhelmingly impressed” by ChatGPT’s capabilities after the program gave him accurate responses to a number of questions he asked as a test.
Crabill told the court he had only been working on civil litigation matters for three months when he filed the motion after he was hired at the Baker Law Group in February. Before then, Crabill, who became a lawyer in October 2021, had worked as a deputy district attorney in the 4th Judicial District for a year and a half. In his affidavit, Crabill said ChatGPT was appealing since he was unfamiliar with civil litigation practice.
“Based on the accuracy of prior validated responses, and the apartment accuracy of the case law citations, it never even dawned on me that this technology could be deceptive,” wrote Crabill. “In short, the initial confirmatory searches emboldened my confidence in the technology and I imprudently accepted the case law research that followed without investigation into each case citation’s accuracy.”
Crabill insisted the use of the made-up case law was unintentional and “not willful misconduct,” statements that Baker Law Group’s senior litigation attorney Robert Harper backed up in an affidavit filed with the court.
“From my familiarity with his work, I do not believe that Mr. Crabill would ever intentionally attempt to mislead the Court with respect to his inaccurate citations,” Harper wrote.
Alongside his affidavit, Crabill filed a screenshot of messages between him and a Baker Law paralegal from the morning he was due to appear in court.
In the screenshot, Crabill wrote “I think all my cases cited from chatGPT are garbage … I can’t even find the cases in Lexis…” The paralegal asked if Crabill had double-checked the citations before he filed the motion. “No. like an idiot.” Crabill responded.
When asked if he planned to withdraw the motion, Crabill said he didn’t know what to do and couldn’t find any case law to support his position. “I don’t have time for this…” The final message from Crabill in the screenshot reads.
Crabill insisted in the affidavit he didn’t know what to do after realizing his mistake.
“I was unaware of what to do in that situation and I was unaware of my ability and obligation to withdraw the motion due to the inaccuracies. In hindsight, the first thing I should have done when your honor took the bench was move to withdraw the motion and request leave to refile after curing the inaccuracies,” wrote Crabill.
Baker Law Group didn’t respond to Law Week Colorado’s request for comment or to specific questions about if Crabill is still with the firm, if the firm had taken any disciplinary action related to the motion or if the firm has any policies around the use of ChatGPT. Law Week was unable to get in touch with Crabill.
The firm’s website doesn’t list the Crabill under its “Our Team” tab and he officially withdrew as counsel on the case on June 9.
On June 20, a paralegal with civil defense firm SGR LLC requested a full transcript of the May 5 hearing, listing “analysis for OARC complaint” as the reason for the request.