From AI Hallucinations to Human Accountability: Rethinking Professional Liability in Legal Practice

Aiman Zahra , B.A.L.L.B

Gilani Law School, BZU, Multan

Introduction

If an AI system fabricates a legal authority, who will bear the legal liability? The AI developer, the lawyer, the judge or all of them? The issue was first clearly addressed in Mata v. Avianca case which established the principle that while AI may generate fabricated citations, the legal responsibility remains firmly with the individual using it (Mata v. Avianca, Inc., 2023).  AI hallucination is a phenomenon where, in a large language model (LLM) often a generative AI chatbot or computer vision tool, perceives patterns or objects that are nonexistent or imperceptible to human observers, creating outputs that are nonsensical or altogether inaccurate. (IBM 2023). AI hallucinations can’t be eliminated through ordinary diligence, as they stem from the predictive language model, which generates text based on statistical associations instead of verifying legal and factual accuracy.

It is not an erroneous pattern that can be avoided with ordinary due diligence but it is a strategically generated, motif matching text but not a fact verification. A public tracker maintained by Damien Charlotin records approximately more than 1750 cases worldwide addressing AI generated fabrication (Charlotin, 2026; HAQQ Legal AI, 2026). In April 2025, Pakistan’s Supreme Court addressed the problem for the first time in Mushtaq Ahmed v. Ishfaq Ahmed Case in which it has emphasized the risk of fabrication of legal references. 

This Article examines the legislative gap which is left unaddressed by the above judgment as it imposes the verification duty on judges without imposing a corresponding obligation on lawyers relying on fabricated and unauthenticated citations which are facilitating the incorporation of AI Hallucinations into court proceedings. Moreover, what Pakistan actually needs is a specific and well-structured roadmap in addition to the principles laid down by judgment for the required legal framework governing AI hallucinations.

 

The Global Judicial Response to AI Hallucinations.

Courts worldwide have converged on one doctrine rapidly the duty to verify AI-generated content before submitting it to a court is treated as absolute and non-delegable (HAQQ Legal AI, 2026). In March 2026, a Sixth Circuit panel fined two attorneys $15,000 each and ordered full reimbursement of opposing counsel’s fees for briefs containing more than two dozen fabricated citations. In the United States the penalties have climbed from four figure fines to $15,000 per attorney in a federal court of appeals and the first bar suspensions tied to AI filings (GC AI, 2026). Courts have punished concealment more severely than the error itself. The Attorneys who denied using AI or blamed their subordinates received the harshest penalties (New York State Bar Association, 2026; GC AI, 2026). The Pakistani Supreme Court was clearly aware of this global trend. Therefore, the judgment expressly surveys the Illinois Supreme Court's AI policy (effective January 2025), England and Wales' 2023 judicial guidance, the Canadian Judicial Council's 2024 guidelines, and Singapore's 2024 guide emphasizing that judges and court users ultimately responsible for AI content and judicial decision, alongside reported sanctions decisions from the United States (Ishfaq Ahmed v. Mushtaq Ahmed, 2025, paras. 11, 14).

 

Ishfaq Ahmed v. Mushtaq Ahmed: The Court's Approach to AI in the Judiciary

The underlying dispute was mundane. A registered sale deed executed in 1999 established the petitioner’s lawful ownership of the property which was subsequently leased to his brother under tenancy agreement in 2016. When the brother defaulted in rent payments, the petitioner initiated eviction proceedings in 2018. Before the matter reached the Supreme Court, it passed through multiple levels of judiciary following the dismissal of the eviction petition and its subsequent reversal by the appellate court. The High Court then set aside the appellate judgment, and ultimately Supreme Court restored the appellate court’s decision in favor of petitioner. The proceedings were prolonged for seven years (Ishfaq Ahmed v. Mushtaq Ahmed, 2025, paras. 2–5). This delay in resolving the dispute and Pakistan’s persistent judicial backlog urged the Justice Mansoor Ali Shah to consider the potential role of AI in judicial reforms. The discussion was anchored in Articles 10 A and 37(d) of the Constitution of Pakistan, which emphasized the Constitutional guarantees of fair trial and expeditious justice (Ishfaq Ahmed v. Mushtaq Ahmed, 2025, para. 7).

The most significant fact for the professional liability scholarship is Judge GPT. It is an AI tool which is specifically built for Pakistan’s Judiciary through collaboration between Federal Judicial Academy and ETH Zurich which is already in use of approximately 1500 Judges of the District Judiciary for research and drafting purposes (Ishfaq Ahmed v. Mushtaq Ahmed, 2025, para. 9). The judgment further refers to a notable domestic instance of judicial AI use, citing two decisions of Additional District and Sessions Judge Amir Munir in which ChatGPT assisted in a civil appeal and a pre-arrest bail matter. The Court identifies these decisions as the immediate impetus for its broader discussion on the integration of artificial intelligence into Pakistan's judicial system (Ishfaq Ahmed v. Mushtaq Ahmed, 2025, para. 8, n. 5). Addressing accountability, the Court draws a distinction between the responsibilities of AI developers and judicial officers, holding that while developers may be liable for technical defects or algorithmic bias, ultimate responsibility for judicial decisions remains with the judges who rely on AI-generated outputs (Ishfaq Ahmed v. Mushtaq Ahmed, 2025, para. 11(iii)). 

 

Closing Gaps in the Legal Framework: Developing a Workable Standard

Ishfaq Ahmed v Mushtaq Ahmed (2025) addresses a smaller problem that it purports to solve. The Supreme Court's duty to check AI output applies to judges who use generative AI to help write judgments. It doesn't extend to lawyers who file documents on behalf of the same courts using AI tools, but it’s the lawyers who are the primary cause of nearly all the sanctions cases in this survey for filing documents with bad AI (Charlotin, 2026). Although the Court allocated responsibility between AI developers and human users, the ruling remained limited to judges. In fact, in a concurring opinion, some foreign courts even now require lawyers to certify that they have reviewed all AI written text prior to filing it. However, the Court considers this to be a reality in other countries, it does not apply this as a rule for lawyers in Pakistan (Ishfaq Ahmed v. Mushtaq Ahmed, 2025, para. 11(ii)). This lacuna would mean that a forged citation by a Pakistani lawyer would probably be subject to the general principles of negligence as defined in the Pakistan Penal Code 1860. This law is not for the case of an unclear AI tool that caused blunders (Khan, 2026).

This was the Court's decision in 2025. But the Regulation of Artificial Intelligence Act, first brought to the Senate in 2024, is still waiting for approval today (BABL AI, 2025). At the present time, there is no specific law in Pakistan regarding Artificial Intelligence. Thus, the judgment remains in a legal void, unaccompanied by any statute that will follow it and fill it out.

There are three things that can help close this divide. The Court has already given the basic tools needed. But it hasn't assembled them. Firstly, the Pakistan Bar Council should stipulate that lawyer ensure that the output of AI is checked. This should be an evident part of the lawyer’s competence. Secondly, the rule adopted by the Court namely that developer is responsible for technical issues, and user is responsible for over reliance on the tool which creates the difficult question of causation when it comes to AI inside. This rule is not based on how the AI failed, but whether the lawyer verified its output before filing. Third, the guidelines ordered under paragraph 18 should be expanded, to also apply to lawyers and not only judges. Even if there is a hallucination scandal in Pakistan, it should not wait around for it. Neither did other countries did as more than 300 federal judges in the United States have already had standing orders regarding the use of AI in court filings, states PlatinumIDS (2026). 

Conclusion

Ishfaq Ahmed v. Mushtaq Ahmed is a genuinely significant judgment, it is researched to a standard comparable with the leading common law guidance documents it cites, it names Pakistan's own AI tool and its verification obligations with unusual specificity. What it does not do is extend that principle to the professionals whose AI assisted filings, not judicial tools, have generated the overwhelming majority of the world's 1,700 plus hallucination cases. Given that no Pakistani case has yet tested professional liability for a hallucinated citation, the space between the judgment's judicial verification duty and an equivalent duty for the bar is not an academic gap, it is a liability vacuum awaiting its first Pakistani test case. The Bar Council should not wait for that case before acting.

References

BABL AI. (2025, August 26). Pakistan Senate proposes AI regulation bill with heavy penalties. https://babl.ai/pakistan-senate-proposes-ai-regulation-bill-with-heavy-penalties/

Charlotin, D. (2026). AI hallucination cases database. Retrieved July 13, 2026, from https://www.damiencharlotin.com/hallucinations/

GC AI. (2026). AI hallucination legal cases: A sanctions tracker (2026). https://gc.ai/blog/ai-hallucination-legal-cases

HAQQ Legal AI. (2026). AI hallucinations in law: 1,313 court cases and counting. https://haqq.ai/blog/when-ai-lies-to-the-court

IBM. (2023, September 1). What are AI hallucinations? IBM Think. https://www.ibm.com/think/topics/ai-hallucinations

Ishfaq Ahmed v. Mushtaq Ahmed, PLD 2025 SC 582.

Khan, M. (2026, March 1). Rethinking liability for artificial intelligence and autonomous systems in Pakistan. LEAP Pakistan. https://leappakistan.com/rethinking-liability-for-artificial-intelligence-and-autonomous-systems-in-pakistan/

Mata v. Avianca, Inc., No. 22-cv-1461 (PKC) (S.D.N.Y. June 22, 2023).

New York State Bar Association. (2026). Beyond the mirage: Beware of generative AI and hallucinations. https://nysba.org/beyond-the-mirage-beware-of-generative-ai-and-hallucinations/

PlatinumIDS. (2026, April 2). 1,227 fabricated citations and counting: Inside the AI hallucination crisis hitting courts worldwide. https://blog.platinumids.com/blog/ai-hallucination-crisis-courts-2026