SecurityLM

Inspiration

I spent the summer working at a long term care facility. Our IT department was struggling to keep up with tickets and wanted to be able to use AI to fill in the gap, but was unable to because of HIPPA and PII violations.

What it does

SecurityLM is a locally sourced SLM platform that allows users to design agents that process and review documents to come to conclusions. It has local document storage and hosts the LLM exclusively on the EmpireAI GPU to ensure maximum security and avoid greater exposure to the internet.

How we built it

We spent a lot of time brainstorming to create what we believe is an intuitive UI and outlined the functionality of the final product. We then utilized Chatgpt to "vibe code" the majority of the project based off of our input and guidance.

Challenges we ran into

We found that the SLMs we used threw errors when they were unable to find the correct part of the text. Instead, instead of offering errors when it cannot find the correct parts of the text, it offers just the document it believes it should be taken from (its usually right).

Originally we wanted to use a software named Ollama to have 3 different SLMs depending on the processing power vs speed that is required. We found that it was impractical under the time constraints, so instead we pivoted to use a single SLM that has a different token allotment depending on the speed selected, offering a similar result.

We struggled a lot with the total integration of all the different pieces. At the start our team divided into 3 subgroups: Frontend, Backend, SLM Operation. Upon the initial completion of these individual pieces, actually combining them proved difficult. It took a while to get the buttons on the UI to map properly to our API, then make sure the SLM was configured properly. In the end it took significant editing of the pieces to make them come together as a whole.

Accomplishments that we're proud of

Successful Integration of the UI to SLM pipeline: It took a long time to get the pipeline of UI button -> FastAPI -> EmpireAI -> Qwen -> Processing -> Qwen -> EmpireAI -> FastAPI -> UI Display to work from start to finish.

Organization as a Team: Teamwork is hard. It took us some time to figure our how to divide the work properly and how to work in tandem. Whether it was dividing the research into ideas up, or designing individual parts of the project, we spent a lot of time communicating to ensure we'd be happy with our final result.

Completion of the project: We knew going into this that it would be rather ambitious to try to tackle this project, especially considering that most of us had never worked with an SLM before.

What we learned

As mentioned above, this was the first time most of us had worked with SLMs. We learned a ton about their pros and cons compared to LLMs, and how to implement them in a larger project. We also learned a lot about the balance to be found with processing power and speed.

Teamwork is hard. We are used to the more structured group work classes give. It was awesome to have the experience to figure out how to work together all on our own. The most important takeaway for us is the importance of constant communication, we found it was a really great way to bounce ideas off each other and keep our project on track!

Integration of various software. We spent a lot of time trying to figure out how to make the various software work with each other in a seamless matter from the user perspective. We found that making flowcharts really helped.