RuleTrace

• 

  RuleTrace interface for comparing two rules and starting an analysis.

• 

  RuleTrace identifies a potential conflict and traces the reasoning back to evidence from both rules.

• 

  1. RuleTrace surfaces supported consequences and questions that still need verification.

Inspiration

Rules often overlap across different sources, such as university regulations, instructor policies, competition rules, workplace policies, and platform terms.

The challenge is not always finding a rule. It is understanding how two rules relate, where a potential conflict comes from, what practical consequences may follow, and what information is still missing.

I built RuleTrace to make that reasoning process more explicit and evidence-grounded.

What it does

RuleTrace compares two user-provided rules and classifies their relationship as:

• consistent
• supplemental
• exception
• potential conflict
• insufficient information

Instead of only returning a label, RuleTrace also provides:

• a concise summary,
• evidence quoted directly from the supplied rules,
• practical consequences supported by the text,
• and questions that still need to be verified.

RuleTrace is designed for issue spotting, not for making a final legal determination.

How we built it

The frontend is built with HTML, CSS, and JavaScript.

The backend uses Python, FastAPI, and Pydantic. RuleTrace sends the two rules to the OpenAI API and uses Structured Outputs to return results in a fixed schema.

The analysis engine is constrained to treat rule text as data rather than instructions, avoid inventing laws or authorities, avoid unsupported consequences, and ground potential conflicts in quotations from the supplied rules.

AI tools were also used during development for architecture discussion, debugging assistance, and implementation guidance.

Challenges we ran into

One major challenge was preventing the model from turning rule comparison into an unsupported legal conclusion.

Another challenge was distinguishing real downstream consequences from simple paraphrases of the rules.

I also had to handle structured AI responses safely in the frontend, including empty fields and differences in returned data formats.

Accomplishments that we're proud of

RuleTrace now provides an end-to-end workflow from rule input to structured analysis.

The prototype can:

• classify five types of rule relationships,
• trace analysis back to specific evidence,
• surface supported consequences,
• identify unanswered verification questions,
• and clearly communicate uncertainty.

What we learned

This project helped me learn how to combine LLM reasoning with structured schemas, backend APIs, frontend rendering, and evidence-grounding constraints.

I also learned that building a useful AI system is not only about getting an answer from a model. It also requires defining what the model is allowed to claim and how uncertainty should be represented.

What's next for RuleTrace

The current prototype focuses on pairwise rule analysis.

Future versions could expand toward:

• document-level rule extraction,
• multi-rule conflict analysis,
• trusted-source rule retrieval,
• rule conflict graphs,
• source and authority metadata,
• rule version tracking,
• and benchmark-based evaluation.

The long-term goal is to help users navigate complex rule systems while keeping the underlying evidence visible and verifiable.

Built With

• css
• fastapi
• github
• html
• javascript
• openaiapi
• pydantic
• python