Privacy Guardian

Detect. Redact. Protect

Comment

Inspiration

The inspiration for Privacy Guardian came directly from my fascination and concurrent concern with the explosion of Generative AI. Tools like ChatGPT are revolutionary, but I couldn't shake a fundamental question: What happens to our data when we ask a question?

The idea that personal, sensitive information—a name, an NRIC, a private address—could be bundled in a prompt and sent to a third-party cloud service felt like a critical vulnerability in this new AI-driven world.

When I saw the Privacy Meets AI hackathon, it felt like the perfect opportunity to address this. I wasn’t against AI; I wanted to find a way to interact with it safely. The vision was to create a digital shield—a tool that would stand guard and ensure the user, and only the user, remains in control of their data.

What it does

Privacy Guardian is a full-stack, on-premises application that acts as a secure shield between a user and any cloud-based LLM service. In real-time, it scans user prompts for Personally Identifiable Information (PII) and redacts it, replacing sensitive data with anonymized placeholders.

Its core function is its privacy-first design: all detection and redaction happen locally on the user's device or within a trusted environment. It does not use any external APIs or online LLMs for its critical function, which guarantees that sensitive user data never leaves the secure perimeter.

How I built it

I approached this project with a full-stack mindset, focusing on a powerful backend and an intuitive frontend.

The Backend Foundation

  • Built with Python + FastAPI for high performance and asynchronous capabilities.
  • Essential for a responsive, real-time tool.

The Core Engine

  • Regex Engine: Precise detection for structured patterns (e.g., emails, passport numbers).
  • spaCy NER Model: Flexible detection for general, unstructured entities (e.g., names, locations).
  • Context-Aware Capture Layer: Uses keyword cues to dramatically reduce false positives.

Making it Smart

  • Advanced Adjudication Logic: Introduced a specificity score system to intelligently prioritize the most accurate match when detections overlap.
  • Intelligent Entity Merging: Combines fragmented results (e.g., "Dr." + "Evelyn Reed") into a single, cohesive entity.

The User Interface

  • Built with React + TypeScript.
  • Simple workflow: paste text, click a button, and immediately see sanitized output.

Challenges I ran into

  • Balancing Precision and Recall

    • Regex = precise but misses many cases.
    • AI = higher recall but prone to mistakes.
    • Solution: a hybrid, multi-layered engine that leverages the strengths of both.

  • Overlapping Detections

    • Multiple layers often detected the same entity.
    • Solved by creating a specificity-based adjudication system with a clear hierarchy of trust.

  • Real-World Messiness

    • PII is rarely clean (e.g., names with titles, fragmented addresses).
    • Required iterative entity-merging logic to ensure natural, correct redactions.

Accomplishments that I'm proud of

  • The Multi-Layered Engine: A hybrid system that is more robust and accurate than any single method alone.
  • The Context-Aware Capture Layer: A novel feature developed to solve false positives in pattern matching.
  • 100% On-Premise Design: Entire application runs locally, with no external API calls for its core function.
  • A Complete Full-Stack Application: Delivered a functional prototype with a performant backend and responsive frontend within hackathon timeframe.

What I learned

This project was an incredible learning experience.

Technical Takeaways

  • Gained practical experience in NLP by architecting a hybrid system that combines rule-based and ML techniques.
  • Built a full-stack system, connecting a performant FastAPI backend with a React frontend.

Conceptual Takeaways

  • Reinforced the belief that privacy and AI can coexist.
  • Demonstrated that on-premise processing and thoughtful architecture can enable innovation without sacrificing data security.

What's next for Privacy Guardian

  • Integration of Fine-Tuned Models

    • Pipeline ready for models like DeBERTa-v3.
    • Plan to integrate custom-trained models into the engine as high-accuracy alternatives to spaCy.

  • Broader PII Support

    • Expand JSON configuration to support more international PII types for global usefulness.

  • Easy Deployment

    • Containerize with Docker to create a simple, self-contained deployment package.

  • Browser Extension

    • Long-term goal: evolve into a browser extension that automatically redacts PII before submission on any website.

Built With

  • and-get-/status.-no-external-or-cloud-based-apis-were-used-for-the-core-functionality.-platforms
  • css3
  • datasets
  • fastapi
  • get-/health
  • git
  • npm
  • pandas
  • pydantic
  • python
  • python-dotenv
  • pytorch
  • react
  • restapi
  • scikit-learn
  • seqeval:-libraries-supporting-the-model-fine-tuning-and-evaluation-pipeline.-apis-internal-rest-api:-the-project-uses-its-own-self-hosted-api-with-endpoints-like-post-/redact
  • services-&-databases-cloud-services:-none.-a-core-feature-of-your-project-is-its-on-premises-design
  • spacy
  • transformer
  • typescript
  • uvicorn
  • vite
Share this project:

Updates

Submission history