ANGEL'S PROTECTION

Our project enhances security in given premises and empowers security officers by using Artificial Intelligence to serve and protect.

Comment 2

Inspiration

Safety is our top priority in an increasingly uncertain world. While security cameras are prevalent, their effectiveness is often limited, requiring manual review of footage to identify specific individuals. This gap inspired the development of ANGEL'S PROTECTION, a cutting-edge solution that enables users to locate individuals quick using natural language queries.

What It Does

ANGEL'S PROTECTION processes live video feeds, storing data based on individuals’ clothing rather than facial recognition, thereby prioritizing privacy. When an authorized user like security guard submits a query, the system utilizes a language model to provide a swift and accurate response.

How We Built It

Our system integrates multiple camera feeds that deliver live footage to advanced machine-learning models. We employ YOLOv11 (with custom dataset and trained from the ground up on Intel AI Developer Cloud) for real-time human detection, coupled with segmentation model to extract clothing features from detected individuals. The classification then process analyzes the clothing attributes—such as color and sleeve length—ensuring comprehensive data collection.

Once the data is processed—thanks to the high-speed capabilities of Intel's AI PC—it is securely stored in InterSystems' vector database. Authorized users can then input natural language queries, which are parsed through a LLM (Llama 3.2) to refine search results before returning the relevant data.

All components of the system are powered by Intel. From the model training on Intel AI Developer Cloud, to model inferencing on the Intel's AI PC, with the Ollama deployed and our other microservices that make it keep up. Intel's AI PC can handle anything that we can throw at it.

Challenges We Encountered

We faced several challenges throughout the project:

  • Model Training: As a first-time endeavor in training models from scratch, we grappled with the complexities of configuring Intel's GPU/XPU. Fortunately, with the guidance of Intel's documentation and support from our mentors, we overcame these initial hurdles.
  • Color Detection Issues: We encountered issues with the color classification models that occasionally misidentified clothing colors due to using a too simple K-Cluster. If we have more time, we could create a neural network model to help us.
  • Vector Database Learning Curve: Working with InterSystems’ vector database and making embeddings are new to us, particularly in understanding its structure and functionality. But we were able to pull through and enjoy using it for our LLM needs
  • Prompt Engineering: Fine-tuning the prompts for a small parameter Llama model proved to be a complex task.

Accomplishments We're Proud Of

We successfully connected a frontend built with React to three different backends:

  1. Image processing
  2. Saving images to the database
  3. LLM-based natural language processing

We also successfully with training and inferencing using the model:

  1. Chaining 3 different model for response
  2. Used a model out of the box and have it work on first run
  3. Trained a segmentation model with a custom clothes dataset(DeepFashion2) and have it ran on detected people.

This integration involved multiple programming languages, including Python and TypeScript.

What We Learned

Through this project, we gained valuable insights into several technologies, including:

  • Iris Vector Database by InterSystems
  • React + Vite
  • Express.js
  • TypeScript
  • Tailwind CSS
  • Python
  • DevOps for connecting multiple system together
  • OpenVision, Roboflow, IPEX LLM

We also expanded our knowledge in DevOps practices to streamline the setup and interconnectivity of our components, and learned about using a reverse proxy for public API endpoints alongside tools like Roboflow, Supervision, and Ultralytics.

We also learned about using the InterSystem's Iris, really useful for us to have a nice and easy to query embeddings that could be used for context system.

What's Next for ANGEL'S PROTECTION

Moving forward, we aim to enhance ANGEL'S PROTECTION by incorporating additional features, such as:

  • Identifying individuals wearing hats, glasses, bags, and other accessories to broaden our search capabilities.
  • Identifying dangerouse momments that require swift response
  • Refining our training processes to improve the accuracy of our outputs significantly.

Built With

Share this project:

Updates