When AI Goes Wrong: The Pitfalls of Relying on AI in the Legal Field

In a recent lawsuit against Avianca, an airline company, the use of artificial intelligence (AI) in legal research led to a major mishap. The lawyer representing the plaintiff relied on an AI program called ChatGPT to prepare a court filing, only to find out that the AI had invented all the citations and quotations in the brief. The lawyer, Steven A. Schwartz, admitted his mistake and expressed regret for relying on the AI program without verifying its authenticity. The case has raised concerns about the reliability and potential dangers of using AI in the legal profession.

The lawsuit, filed by Roberto Mata, claimed that he was injured when a metal serving cart struck his knee during a flight to Kennedy International Airport. Avianca's lawyers requested the case to be dismissed due to the statute of limitations, but Mata's lawyers vehemently objected, citing various court decisions to support their argument. However, the cited cases and quotations could not be found because ChatGPT had generated them.

Judge P. Kevin Castel, presiding over the case, described the situation as "an unprecedented circumstance" and scheduled a hearing to discuss potential sanctions against the lawyer. This incident highlights the ongoing debate among legal professionals regarding the value and dangers of AI software like ChatGPT. Lawyers are grappling with the need to verify the information provided by AI and prevent misleading or false submissions in court.

Stephen Gillers, a legal ethics professor at New York University School of Law, emphasized the importance of not blindly accepting AI-generated output. He cautioned that cutting and pasting AI-generated content into court filings is not a responsible approach. The legal community is now actively discussing ways to avoid similar situations and ensure the accuracy of information derived from AI systems.

While this case exposes the risks associated with AI in the legal field, it also serves as a reminder that human expertise and critical thinking are still essential. The complexity of legal research and the nuanced understanding required to construct compelling arguments cannot be replaced by AI alone.

Tips for Fact-Checking AI-Generated Data

As the use of AI becomes more prevalent in various fields, including legal research, it is crucial to implement effective fact-checking measures to ensure the accuracy and reliability of AI-generated data. Here are some tips to consider:

  1. Independent Verification: Always verify the information provided by AI systems independently. Relying solely on AI-generated output without cross-referencing it with reputable sources can lead to misinformation or errors.

  2. Use Trusted Databases: Consult reliable legal databases and resources to verify the existence and authenticity of cited court decisions and quotations. Double-checking the information against trusted sources can help detect any discrepancies.

  3. Human Expertise: Leverage human expertise and critical thinking in conjunction with AI. Lawyers should actively engage in the research process and review the AI-generated content to ensure its accuracy and relevance.

  4. Legal Professional Networks: Seek advice and insights from fellow legal professionals who have experience working with AI tools. Sharing knowledge and experiences can help navigate the potential pitfalls of using AI in legal practice.

  5. Ethical Considerations: Reflect on the ethical implications of relying on AI-generated content. Consider the potential consequences of submitting inaccurate or misleading information to the court and prioritize ethical responsibility in legal practice.

By adopting these fact-checking practices, legal professionals can harness the benefits of AI technology while mitigating the risks of relying on potentially erroneous or fabricated information.