Sentinel-Edge AI

Sentinel-Edge: Air-gapped AI security auditor running 100% on-device. It scans code and contracts for risks via Llama 3.2 3B on the Snapdragon X Elite NPU. Zero network calls, total privacy.

Comment

Summary

Enterprises in legal, defense, and finance cannot use AI on their most sensitive documents because every cloud LLM is a data leak risk. A lawyer cannot paste a client's contract into ChatGPT. A defense engineer cannot run their code through GitHub Copilot cloud. So they do nothing and stay unproductive.

Sentinel-Edge AI is an air-gapped AI security auditor that runs 100% on-device. Upload a legal contract or source code file - it detects risks, flags vulnerabilities, and generates a full report without a single network call.

Powered by Llama 3.2 3B on the Snapdragon X Elite Hexagon NPU, every byte of your sensitive data stays on the machine. (Qualcomm Track)

Inspiration

We wanted to make an AI that is truly the epitome of privacy. Safeguarding your info and scouting out possible vulnerabilities in a knowledgebase, from anywhere, at anytime, with no internet. These are not features we have seen in any other LLM as there is always a third party service involved that can access your data.

What it does

Sentinel-Edge is an air-gapped AI security auditor. You upload a PDF contract or source code file, and it returns a color-coded risk report: HIGH, MEDIUM, and LOW findings with excerpts and recommendations. It detects legal risks like one-sided IP assignments and missing liability caps, and code vulnerabilities like hardcoded secrets, SQL injection, and broken authentication. No internet. No cloud. No data leaves the device.

How we built it

We built a full RAG pipeline on top of the Snapdragon X Elite NPU. Documents are chunked and embedded using sentence-transformers (all-MiniLM-L6-v2), stored in a local ChromaDB vector database, and queried at inference time to provide relevant context. Llama 3.2 3B runs via the locally hosted AnythingLLM, offloading compute to the Snapdragon x Elite NPU. A FastAPI backend handles job orchestration and a Streamlit frontend polls job status and renders findings. The knowledge base covers 10 legal risk categories and 6 code vulnerability types, all prebuilt and shipped with the app.

Challenges we ran into

We initially intended to host the LLM with the Nexa SDK, but we ran into many errors trying to call the LLM from a python script even though the CLI was seamless. Also, we attempted to make our frontend with react, and nodeJs, but the node dependencies involved upgrading python and therefore disrupting our entire pipeline as it relied on a more stable python3.11 while node installed python 3.14.

Accomplishments that we're proud of

Despite our technical setbacks, we shipped a clean, functional UI using Streamlit and got the Snapdragon NPU running inference through AnythingLLM on our first attempt with edge hardware. Both of those came together faster than we expected.

What we learned

We learned that "offline-first" is a design constraint that has to be enforced from day one as it touches dependency management, model loading, and data persistence in ways that are hard to plug and play later by porting away from a cloud GPU.

What's next for Sentinel-Edge AI

I think next steps for us would be to try to use better models, and refine our prompt engineering tailored to each client. We would also like to deploy these models at scale with larger databases of context while maintaining its edge-device offline functionality

Built With

  • anythingllm
  • chromadb
  • langchain
  • llama3.2-3b
  • pdfplumber
  • python
  • sentence-transformers
  • snapdragonxelite(hexagonnpu)
  • sqlalchemy
  • streamlit
Share this project:

Updates

Submission history