Inspiration
There's already a lot of anomaly detection for security applications in cybersecurity, but not enough in prompts for LLMs.
What it does
SAnDL (Sophisticated Anomaly Detection of LLMs) runs a prompt through two models: first an embedding encoder to turn strings into vectors. These are then fed into a series of Isolation Forests to output a probability that the given prompt is malicious.
How we built it
We built the frontend with Streamlit and the backend with Flask. The embedding is done by a sentence transformer model from hugging face (a variant of BERT). In order to produce the probabilities, we input our embeddings into multiple isolation forests and then averaged the results. We hosted a server and a Postgres database on Google Cloud. The server handles requests and prompts. For our cache of prompts, we put the prompts through a tfidfvectorizer and then stored these vectors within a Redis cache. Login information and API keys are stored within the Postgres database, which the server is able to query and receive data from.
Challenges we ran into
We ran into multiple challenges with our data set, particularly with the large size of the data set and how slow converting the text data to vectors is. We were unable to work with the entire data set like we wanted to in the beginning due to computational cost. In addition, we had troubles deploying to Google Cloud due to us finding the documentation was unclear in cloud run regarding the use of other services with it. For frontend, there was some struggle in learning Streamlit so quickly and connecting frontend and backend. Lastly, we had some trouble with finding the right word to vector algorithms.
Accomplishments that we're proud of
We're proud of our success in time management and understanding the scope of the project. We began this weekend knowing that our idea was a bit ambitious and quickly saw this manifest with the challenges we quickly encounter. However, we managed to be quick on our feet and able to adjust. Given the time and data constraints, we are proud to have been able to complete a fully functional web application with a clean and intuitive UI. Additionally, we were able to gain exposure to and learn about a number of technologies that were fairly new to us, such as Google Cloud and Redis, which was very rewarding
What we learned
We had to learn many new technologies, especially involving deploying our app to production, especially in using Google Cloud and Docker in conjunction. Additionally, we learned a lot about using Redis cache combined with sklearn to perform lookup in the cache in an efficient manner. For frontend we learned how to use Streamlit from scratch. Lastly, we researched a lot of different unsupervized or semisupervized learning techniques as well as different ways to convert text to vectors during our search for a fitting algorithm.
What's next for SAnDL
While we do think that we achieved a lot over the span of this hackathon, there were definitely a lot of potential areas for improvement as well. The most glaring issue to us is that we were only able to train on a small portion of our data set for our models, which likely detracts from the performance greatly. In addition, while the data set we had was very large, we ended up having doubts about the quality of the data set a bit later in the project, although by then it was too late to look for a new data set, rerun the sentence transformer, and retrain the model. For the prompts, there were many different methods of creating vectors from the text data. Although we tried some of them, we would have loved to have been able to test out more of them.
Log in or sign up for Devpost to join the conversation.