SentinelLM

Your LLMs Monitored and Protected

Comment

Inspiration

LLMs produce a lot of content very well. But if they are given contexts that are too large or questions they aren't supposed to answer, LLMs falter like humans do: by blabbering things or messing up the formats.

Working in big tech I saw that our reliance on LLMs for customer-facing tools is growing. Still, we do not have a standard set of tools to deal with the problems that arise when the model does not do exactly what we expect, or worse does something completely different.

We can use LLMs to build the next generation of apps and products, but we need to keep them on the rails for them to be as useful as we hope.

What it does

SentinelLM is a set of filters and cron jobs that allow developers and AI teams to run health checks on their LLMs, or programmatically check every input and output from their LLMs, to make sure that their customer experience is safe and secure.

How I built it

The Backend

Golang and SST.dev are the 2 tools we lean on heavily. SST provides a great base to create a serverless application with the necessary CDK constructs to build our custom tooling. Golang provides the perfect language features to easily parse and interact with the freeform data that we must handle with models.

The Frontend

Next.js and React allow us to interact with the backend quickly and has a lot of provided magic to help get us moving quickly.

Challenges I ran into

The biggest challenge was not knowing how to allow a variety of users to use the service. For example, we were expecting a very rigid format for headers to come from users. So we only exposed the following option:

{
  "<INSERT-HEADER-FIELD>": "<FIELD-VALUE>"
}

But we found that users should be able to input anything they want into their headers. Validation of this was an interesting challenge!

Accomplishments that I'm proud of

Building the MVP in under 1 month! It was a sprint of development, but a lot of lessons were learned.

What I learned

So much! Here's a list:

  • Cron Jobs on AWS
  • SQS on AWS
  • Prompt Injection and how to handle it
  • Validation Techniques for Text
  • Text Watermarking

What's next for SentinelLM

Getting our first users! We'd love to have people take a look at what we got, take their feedback, and continue to build for developers.

Share this project:

Updates