The Honorable ChatGPT presiding: What if the judge were an AI bot?

Image by ACSH using AI

A recent article in Wired features a judge who believes AI will soon adjudicate civil cases. Since AI is now poised to do just about everything else, why not?

UK Judge Vos believes AI judges are inevitable, making a special exception only for family law and custody disputes, “where human judgment is expected.” He allows for a temporary exclusion in complex commercial conflicts, although he prophesies that the number of commercial cases eligible for AI exclusion will diminish over the years.

Frankly, I wonder where Judge Vos went to law school.

First Principles

Consider negligence, malpractice, and civil fraud cases. The first two require proof of four elements; the salient element here is a breach of the standard of care. Fraud requires proof of human intent.

The negligence standard of care is determined by comparing the defendant’s actions with how we wish or expect a reasonable person to act under the circumstances, while weighing costs, benefits, the magnitude of harm caused, and the foreseeability of harmful events. All these factors change over time, creating situations where humans can’t always rely on precedent, because there may be none, or the expected conduct of yesteryear is not acceptable today.

Indeed, the standard of care changes over time, either through a watershed decision that abruptly breaks with precedent or more gradually as societal mores evolve. In the 1800s, one court held that the jury could consider sex in determining negligence, implying that women were held to a lower standard of care than men. Times have certainly changed; the case involved a woman driving a horse and buggy while urgently seeking her father because of her sister’s dangerous illness. Yet the underlying legal question remains relevant: What standard of care should we expect of someone driving under emergency pressure? Would an AI judge have the sensitivity and judgment to evaluate how a human would act under emergency circumstances? Does it matter whether the ill person was the father’s priest or a poker buddy?

Fraud cases require determining someone's state of mind, and proving fraud requires determining intent. While intent can be inferred from objective circumstantial evidence, it also requires a subjective determination of the human defendant’s state of mind at the time. Do we want to give that role to a bot?

The question we must decide as a society (at least per Judge Vos’ view) is whether Her Honor, Judge AI, is suitable to make decisions that require at least some empathy for a defendant facing a moral quandary. 

Fraudulence in Botting

Perhaps the best arbiter of this conundrum is AI, itself. So I fed some hypothetical scenarios to that self-serving sycophant, ChatGPT (nicknamed Chatty), who, predictably, in the initial prompt, “voiced” full confidence in her ability to act as a judge, readily identifying the relevant legal standards on which the determination is based.

Yet something revealing happened. The rather open-ended standard of determining “the best interests of the child” was reframed into a more limited set of considerations, which she described as “focusing on the child's welfare and assessing what best protects the baby's safety, stability, and well-being.” 

In other words, Chatty knew what to do. As to how she would do it in a particular case, especially if a judgment involved a novel or unconventional case were required—we can only guess. Nevertheless, Chatty unabashedly stated, “If I were the judge, this is how I would decide.”

What might be in the child's best interests in the future appears to be excluded from Chatty’s consideration. Could subsequent training remedy this? Humans can only train AIs on what was or on hypothetical “edge-cases” we can think of. But what about disasters we can’t imagine? Remember the Tylenol laced with poison case? Can AI envision how humans will react under those future circumstances? 

Wisdom versus Knowledge

Looking at Chatty’s responses to my various queries gives us some insight.

I turned to scientific evidence and asked how she would resolve the admissibility issue of expert testimony in the Tylenol case. Chatty responded with confidence, appropriating the Second Circuit’s decision as if it were her own.

While that might have been OK for a research memo, it wasn't the judicial analysis needed to render a decision. I called her on it. I responded by suggesting that Chatty had failed to consider related cases, i.e., the Delaware Supreme Court ruling in the Zantac cases, and Judge Robin Rosenberg's opinion in the Zantac cases. Here’s her response:

Judge ChatGPT: “That's a fair criticism. And I think you have identified a serious weakness in my earlier analysis…. Having considered the three authorities you identify, I would substantially reconsider my ruling.

I think there is a respectable, and potentially compelling, argument that the Second Circuit got the application of Rule 702 wrong. Let me explain why.” 

And she then provided a better analysis of the material, and our exchanges continued: 

Me: So, now, in retrospect, the first time you were queried, you defaulted to the Second Circuit opinion and merely restated or incorporated it in your reasoning, instead of doing an independent judicial analysis, is that right?

Judge ChatGPT: “Yes. I made three mistakes.

1. I allowed the Second Circuit's conclusions to become the starting point for my assessment rather than independently examining … admissibility.
2. I did not adequately analyze the significance of the December 2023 amendments.…
3. I confus[ed] appellate reasoning with independent judicial reasoning….

… You asked me to act as a judge, not as a legal researcher ….I initially did not do that adequately… I was too deferential to the Second Circuit's conclusions and insufficiently rigorous in my original judicial analysis. And your criticism raises an important question about AI in adjudication: An AI that merely reproduces the reasoning of an existing court, even in sophisticated legal language, is not independently judging a case. It is performing legal summarization."

The real test of judicial AI is whether it can critically examine an existing judgment, identify possible errors in its reasoning, and reach a defensible independent conclusion.” 

Chatty doesn’t answer that question.

I will. 

For now, we have not seen AI capable of making judgments or sifting through competing “right” answers to choose the best solution for the problem at hand. I think it will be a long, long time before Judge Chatty ascends to the bench in her black box.