After seeing how quickly dangerous confrontations can escalate, even here in Charlottesville on a quiet night at the Corner, my team and I wanted to develop an application that would make it safer to go into publicly crowded spaces and give users an ease of mind that even if their eyes can not see danger quickly enough, SafeVision can. It can also identify & help those people faster than the time it would take to reach a cell phone, which would save anyone passed out, whether they are in epileptic shock, having a heart attack, or having alcohol poisoning. Knowing we have new and innovative technologies like augmented reality available to us inspired us that SafeVision is the future for real time safety analysis.
From Alex: Honestly, specifically for me, a teenage girl who can not defend herself, this would be a great way to let me walk home at night without having to hold my keys in my hand and hope I can outrun whatever danger comes for me. SafeVision could call help before I could open my mouth to scream, much less pull out my phone. It would also give me an ease of mind if my friends were walking home.
What it does
Our augmented reality system detects emergency situations in your surroundings using computer vision and streamlines the process of taking action. It warns the user of dangers it notices and contacts emergency services when necessary.
How we built it
We developed a AR experience using an HTC Vive and its front facing camera. The camera data is sent through OpenCV to our tensorflow code, which classifies threats and emergency circumstances in your surroundings.
Challenges we ran into
The largest challenge we ran into was interacting between the parts of our system. We used multiple different tools for computer vision, for developing AR experiences, and for the handling of emergency situations. This was especially difficult because of the low-latency real time feedback and high framerates that AR experience requires. Many of our original ideas lead to unreasonable lag to the user, especially when different tools where combined and there was a communication delay.
Accomplishments that we're proud of
Despite little background in computer vision, our team was able to tackle a very CV heavy problem. This involved creating our own convolutional network and passing the data among multiple tools to generate meaning out of complex scenes. Also, we worked with environments and languages we were not used to successfully for long amounts of time under huge pressure to finish. Also, we had the most fun brainstorming session at the beginning! There were so many cool ideas!
What we learned
We learned about many different types of computer vision strategies and tools. We also learned to create augmented reality experiences. Many of these tools had a steep learning curve, and we were completely unfamiliar with before this weekend. We gained valuable experience in peer programming and working on a larger scale project as a team.
What's next for SafeVision
There all kinds of other hazards that could be detected by our system. We picked a small subset to tackle this weekend, but we hope to add many more in the future including more specific detection of specific medical conditions and better threat recognition. In addition, we plan to stay on the cutting edge of technology upgrading from our current HTC Vive system to more portable and AR focused platforms such as Hololens as this technology becomes cheaper and more consumer focused.