PACMAN.ai - Proactive Automated Compliance Management AI

Cloud Compute Cost Breakdown (USD 126 shown)
1.5k Instruction-Tuned Compliance Dataset

Inspiration

Building global products isn’t just about adding new features. It’s also about respecting the laws of the land. A login blocker that works perfectly in Europe could suddenly break the rules for teens in Utah after 10pm. Every country, every state, even every hour can have its own set of rules. Without clear legal context, teams spend countless hours second-guessing compliance. Innovation slows down, and the risk of penalties grows. Our team from NUS, Poland, Ireland, Germany, and Singapore saw this challenge up close in platforms like TikTok. Each new feature has to make its way through a maze of regional laws, from the EU’s Digital Services Act to unique state regulations in the US. Right now, this process is manual, fragmented and full of blind spots. That’s what inspired us to tackle the problem.

What it does

PACMAN analyzes product feature artefacts and determines whether geo-specific compliance logic is required. It returns structured JSON with a compliance flag, the relevant law, and a concise explanation. Results can be exported to CSV, and a React.js web UI presents traceable, reviewable evidence.

For example, when given a feature called “Supervisory Authority Communication”, PACMAN.ai retrieved relevant text from the EU Digital Services Act through Amazon Kendra and flagged that geo-compliance was required. It explained that EU law mandates region-specific communication and reporting duties, meaning enforcement must be limited to EU users. This shows how the system grounds its reasoning in real law and makes compliance decisions explainable.

How we built it

We gathered about 1.5k instruction fine-tuning pairs. Each included a product feature name, its description, and relevant law snippets retrieved from Amazon Kendra via vector search. Positive pairs captured real compliance obligations, while negative pairs covered business-driven rollouts outside the scope of law. Creating this dataset took significant effort and many failed iterations to align features with legal text.

We fine-tuned Microsofts Phi-2 model on AWS SageMaker using g5.4xlarge and A10G GPU instances. Training and testing cost about 150 USD. LoRA weights were stored in Amazon S3, and during inference we invoked the base model with these weights. The endpoint was deployed on SageMaker and validated via an AWS Lambda function to ensure outputs remained stable and did not hallucinate.

Challenges we ran into

Lynx-based UI development proved difficult, especially on iPhone where a Mac was required for proper setup. One teammate spent hours troubleshooting real-time updates, only to face device-to-device networking restrictions on university Wi-Fi. We switched to React.js for faster, more reliable development. On the ML side, we hit repeated networking errors, tensor mismatches, model download failures, and endpoint configuration issues. It took multiple iterations before the training-deploy-evaluate loop stabilized.

Accomplishments that we are proud of

We successfully fine-tuned Phi-2 on AWS and built a working compliance-detection prototype. We integrated Amazon Kendra for legal grounding, automated audit-ready CSV exports, and built a functional React.js UI at pacmanlaw.org. Despite many setbacks, we shipped a system that makes compliance detection explainable and practical.

What we learned

We learned to fine-tune compact models efficiently on AWS while managing costs. We saw the importance of grounding LLMs in external knowledge, using Kendra to connect features to real laws. We also gained experience debugging SageMaker endpoints, testing inference consistency with Lambda, and navigating the practical difficulties of Lynx versus React.js for cross-platform UI development.

What is next for pacmanlaw.org

We plan to expand beyond 1.5k pairs and broaden jurisdiction coverage. We want to combine LLM reasoning with static code and runtime analysis for hybrid detection. We will add a human-in-the-loop review flow and pilot pacmanlaw.org with global product teams so compliance checks become part of every feature rollout.