Horus Eye

We built a preliminary monitoring and surveillance system that alerts a concerned individual if an anomaly is found.

Comment

Inspiration

  • We want to alert on some anomalies, for example, some specific objects removed or unknown person appear in the house
  • We want a summary of events that happened during the day/time window when we are away

What it does

  • Alerts on any anomaly discussed above in near real time
  • Summarizes a period, eg. a day for any abnormal activity like missing objects, unknown intruders in the house or child reaching out to dangerous places

How we built it

  • We used webcam as live video feed
  • We used YOLO World model to detect objects based custom aspects
  • We used Gemini API to summarize batches of videos and create consolidated summary out of subsequent summaries

Challenges we ran into

  • We faced issues in setting up raspberry pi and did not have GPU, and used laptops for the project

Accomplishments that we're proud of

  • Effectively dividing tasks among teammates and creating working prototype in a short time
  • Visiting Baird Point in the morning for sunrise

What we learned

  • Generative AI applications in surveillance and monitoring
  • Object detection and image segmentation using YOLO with custom aspects
  • Summarization using Gemini
  • Accessing webcam and processing real time image feed
  • Teamwork and how to have fun

What's next for Horus Eye

  • Run the model in raspberry pi, attach wheels to it and patrol around the house
  • Build a vision agent where we assign a role each day/time window prompting it to carry specific tasks
  • Use optimal model for small device like raspberry pi for fast inference and less hardware requirement
  • Add actions to the device like talking with the child if it is reaching out to places it is not supposed to
  • Use in other scenarios like surveillance cameras to detect abnormal events

Built With

Share this project:

Updates