Overwatch

The problem we're solving

Walking alone at night in a dimly lit area poses real safety risks. Moreover, Davis has seen many examples of violent crimes in the past few years, from stabbings, gun violence, to even rape. There are existing solutions for ensuring safety, however most of these measures fall short. Take for example, the UC Davis Police Department Safe Ride program. While it does provide for a safe means of individual transport, wait times are often too long and the program has limited range. Another popular solution is calling a friend while walking at night to ensure safe arrival. However, this is only a reactive measure, not a proactive measure. That is, no action is taken until something bad actually happens.

What it does

Overwatch is a drone+phone app combo that serves as a lightweight way to detect and deter nighttime threats (i.e., suspicious people who want to hurt you). You can summon a companion drone from the app to watch over you as you travel to your destination. The drone features a multitude of different cameras and sensors that keep a close eye on the area surrounding you and will alert you if a threat is detected. The drone also serves as a deterrent to people who might want to cause you harm.

How it works

The drone has several cameras attached, including a thermal camera and a GoPro camera. The thermal camera can detect human heat signatures even in the dark. This drone can then take pictures of the user’s surroundings and analyze them using computer vision. The results of this analysis are sent to our mobile application, which informs the user of potential threats.

Our hackathon project demonstrates the important portions of this overall pipeline. We collected thermal camera videos with the drone. This video file is used by a “mock drone” Python program, which streams video frames to an analysis Python program like a drone would. The analysis program locates any potential threats and sends this information to our mobile application over the network. The mobile application displays the location of potential threats to the user.

The analysis program does not attempt to identify any individual personally or even try to guess if someone is suspicious. Instead, it merely notifies the user when a high thermal signature (i.e., a potential human being) gets close to them. The user can then assess the situation around them as necessary.

Challenges we ran into

The image processing was challenging to get right. It was difficult to discern human signatures from other objects accurately. High-absorption objects like sidewalks and bikes would have similar heat signatures to humans.

It was also a challenge to accurately track human thermal signatures as they moved across multiple frames. Thermal signatures can blur together on camera when they come close together.

Accomplishments that we're proud of

We are proud of completing the data flow starting with raw readouts from the sensors onboard the drone, applying computer vision methodologies for image segmentation, and building a fleshed-out mobile application to interface with the data.

What we learned

In developing Overwatch, we gained significant insights into the complexities of integrating hardware and software solutions. We learned the intricacies of interfacing with various drone sensors, including thermal and GoPro cameras, and the challenges of real-time data processing. We also deepened our understanding of computer vision techniques for image segmentation and threat detection. Furthermore, we learned the importance of building a user-friendly mobile application for seamless interaction with the drone's data. This project provided valuable experience in developing an end-to-end system, from sensor data acquisition to user interface design, and highlighted the importance of robust data flow and efficient communication between different components.

What's next for Overwatch

We would like to utilize better hardware for higher quality readings of the surroundings and implement an autonomous following solution, where the drone automatically follows the user around as they walk alone at night.

If we had more time (and the required hardware), we would make the “mock drone” and analysis Python programs run on the drone itself.