RedactLM

Inspiration

Something that has always bothered us is LLMs being used frequently everywhere, both for business and personal use, and people enter sensitive information constantly. As a result, we looked at the concept of LLM routers and we took our own approach by building our own central LLM frontend as well as feeding prompts through both static checks and dynamic local LLM checks.

What it does

The project flags and prevents sensitive information from being sent between employees and proprietary LLM services such as Anthropic Claude, ChatGPT, and Google Gemini. Any LLM service that implements the OpenAI standard is supported. We use a combination of rule-based and regex checks as well as open-source local models to determine whether PII has been shared within a message. We have also built a custom chat interface to centralize LLMs and act as a complete router. Our project can be fully self-hosted in a single docker container plus the database and supports connecting to local LLMs over both OpenAI and Anthropic-compatible APIs.

How we built it

We scaffolded and built the project using Bun.js, TypeScript, Hono (API), Postgres (DB), Tesseract (OCR), Ollama (local models), Docker (containers) for the backend. The frontend uses React and Vite. The initial foundation of the program was handwritten but we later turned to Claude Code in the development process in order to speed up development and deliver a finished product within the deadline.

Challenges we ran into

Writing boilerplate code was time consuming and was prone to mistakes, so we leveraged modern LLMs and Claude Code which considerably sped up the process and left us focusing on the important parts. Another challenge was that large documents took too long to scan and used too many tokens, so we added a token limit for files to scan. Also, our project was initially two repos but we found it was too tedious to manage and deploy, so we converted it into a monorepo.

Accomplishments that we're proud of

We now have a complete product with features such as an LLM chat, reviewing others' chats, adding LLM backends, user management, detection policy options, local-LLM based detection, SSO, blocking custom keywords, and an immutable audit log. Our project is very polished and has a nice UI.

What we learned

This was our first time using AI in development, and we learned how LLMs can greatly speed up the development process. We used a cutting edge Claude Opus 5.5 model, without realizing it, so it was great to experience the frontier of AI first-hand. We also learned that monorepos can be easier to manage. Finally, we learned that redaction is surprisingly complex. Due to time constraints, we opted to only block PDFs with sensitive data rather than redacting as redacting raster images is a nightmare.

What's next for RedactLM

We plan to further polish the backend/API and add Claude Code support in the future. Perhaps one day, we can try to redact PDFs and images as well, rather than just text-based content. We should also compare local models and find the best one for our purpose.