AI Decision Clerk1

Our AI Agent analyzes a draft decision to ensure that the judge’s decision is consistent with legal precedents it cites, enhancing the integrity and efficiency of courts, and boosting productivity.

Comment

Inspiration

Judges rely upon humans, ("Decision Clerks") to draft Decisions, and to ensure that the draft Decision is consistent with caselaw precedents cited in the Decision, before it is signed by the Judge. This project provides an AI Agent that will help a judge know whether the decision they are asked to sign is consistent or inconsistent with the cases cited in that Decision. The AI Agent helps to keep the Decision Clerk diligent and honest. The example of inconsistency between the Decision and the cited cases (e.g. Laba v, Carey) in Prendergast v. Swiencicky, 183 A.D.3d 945 (Third Department, New York, 2020)) proves the AI model's ability and illustrates the urgent need for this AI Agent. This AI Agent also aims to increase the productivity of judges by quickly verifying the consistency of draft decisions with cited cases. The AI Agent streamlines the judge's sign/no-sign decision-making process, potentially reducing the time judges spend reviewing each cited case. This not only enhances individual productivity but also improves the efficiency and fairness of the judicial system as a whole.

What it does

A judge uploads a draft decision and asks the agent to verify its consistency with the cited cases. The agent analyzes the decision and identifies any inconsistencies with cited cases. The Agent provides the judge with a detailed explanation of the inconsistencies, including relevant passages from the cited cases and the decision, highlighting the specific legal reasoning behind its findings. This information empowers the judge to make informed decisions and ensure the accuracy of their legal judgments.

Idea Proof of Concept Script for Identifying Inconsistency with Cited Cases {Results obtained in AIStudio using Gemini 1.5 Pro

Assume the role of an AI Decision Clerk of the Appellate Division 2nd Department. Determine whether there are inconsistence between the Majority Decision in Prendergast v. Swiencicky and the cases cited therein (Laba v. Carey, Addesso v. Shemtob). Make a list table of any such detected inconsistences.

Agent: Of course. Please upload the decision citing the cases. [upload the Decision]

Agent: Processing the information... The Decision is inconsistent with the cases cited. ... Inconsistencies Detected: Prendergast v. Swiencicky As the AI Decision Clerk for the Appellate Division 2nd Department, I have analyzed the Majority Decision in Prendergast v. Swiencicky and compared it to the cited cases, Laba v. Carey and Addesso v. Shemtob. The following table outlines the detected inconsistencies: [To see Table Rows and Columns see https://vimeo.com/953300449 or https://www.youtube.com/watch?v=e_Mwhb5ayXg ] Analysis: The inconsistencies identified suggest that the Prendergast majority may have selectively interpreted or misapplied the cited cases to justify a departure from the strict requirement of a "lien covenant" in the contract. This raises concerns about the reliability of the Prendergast precedent and its potential to create confusion regarding sellers' obligations in real estate transactions.

Recommendation: Given the detected inconsistencies, it is recommended that the Appellate Division review Prendergast v. Swiencicky en banc to provide clearer guidance on the interplay between express contractual terms, industry customs, and the seller's obligation to deliver marketable title at closing.

User: That's concerning. I'll review it further. Thanks for your help.

[AIStudio Gemini 1.5 Pro Alternative Proof of Concept "conclusion"]: Conclusion: The identified inconsistencies suggest that the Prendergast majority may have selectively interpreted or misapplied the cited cases to justify their prioritization of industry custom over the contract's express terms. This raises concerns about the reliability of the Prendergast precedent and its potential to create ambiguity in future real estate disputes. Recommendation: Given the identified inconsistencies, it is recommended that the Appellate Division consider reviewing Prendergast v. Swiencicky en banc (by the full court) to provide clearer guidance on the interplay between express contractual language, industry norms, and the seller's obligation to provide a title free of encumbrances at closing. Clarifying these issues will enhance predictability and fairness in real estate transactions within the Second Department's jurisdiction.

How we built it

This AI Decision Clerk concept and method was proved in Google's AIStudio (based on manual-upload of Decision and the cited caselaw), then replicated in Vertex Agent Builder. (I Watched Google's Youtube tutorials for Vertex Agent Builder, and also used Gemini Pro to make a first draft of the "instructions" and the "tool" input/output YAML code: Example "Instructions" in Vertex AI Builder Agent named Decision_Reader_Agent1:

  • A judge uploads a draft decision
  • Use a Decision Upload Tool to obtain the draft Decision from the user: ${TOOL:draft_decision_uploader}
  • The Agent uses NLP to generate a list of cited cases in the Decision and uses a case retrieval method to obtain the text of each caselaw precedent: The agent analyzes the decision and retreives the cited cases: ${TOOL:local_stored_cases}
  • Utilize natural language processing (NLP) techniques: The agent checks the Decision's consistency with the cited cases. The agent identifies inconsistencies or consistency of the Decision with each cited case. If inconsistency is detected, the Agent provides the judge with a detailed explanation of the inconsistencies, including quoting relevant passages from the cited cases and the corresponding portion of the draft decision, highlighting the specific legal reasoning behind the Agent's findings. This information empowers the judge to make informed decisions and ensure the accuracy of their legal judgments. The agent should leverage NLP models to understand the language of legal documents and identify relevant information in each cited case.
  • The agent should compare the decision's interpretation of cited cases with the actual legal precedent.
  • Provide a user-friendly interface: The agent should be able to interact with judges and legal professionals in a clear and concise manner, presenting its findings in an easily digestible format.

Challenges we ran into

Vertex AI doesn’t directly provide case law or statute databases. However, a user (judge) can integrate external legal databases (such as Westlaw, LexisNexis, or public legal repositories) into the system. [For Demonstration purposes, we manual-upload a sample decision that is known to be inconsistent with the caselaw it cites, and manual-upload sample caselaw (cited therein) into Data Stores: ${TOOL:draft_decision_uploader} and ${TOOL:local_stored_cases} ]

Accomplishments that we're proud of

Our AI Agent revolutionizes the process of review of a judicial draft decision by employing advanced natural language processing to analyze the decision and the cases it cites (and optimally, the prior cases cited in the cited cases, iteratively). This helps to ensure that the judges’ rulings are consistent with legal precedents, and that no one has successfully misrepresented the caselaw, enhancing the integrity and efficiency of court proceedings. By automating the citation-verification task, the AI Decision Clerk Agent allows judges to focus on the facts and the nuanced aspects of each case, thereby boosting productivity.

What we learned

Vertex AI Agent Builder agents are a new way to create functioning applications leveraging the intelligence of Gemini Pro models. However, the fixed AI model (Gemini 1.0 Pro) having limited context length, and the fixed RAG features (Document "Summary" and text "snippet" retrieval) of Vertex AI Agent Builder are not flexible and extensive enough to support the entire-PDF-document retrieval and complete document analysis required implementing for this particular use case. I believe that upon removal these limitations the AI Decision Clerk proven in AIStudio could be fully implemented in Vertex Agent Builder with full automation of whole-document retrieval and agentic analysis of documents.

What's next for AI Decision Clerk1

Adding a Web-based front end and Decision-upload features, plus automated caselaw-retrieval obtained from Westlaw, Lexis, casemine.com, casetext, etc.

Built With

Share this project:

Updates