The topic of security in the field of IoT is super important but can sometimes seem overwhelming and scary. After all, shouldn't you always feel safe at least in your own house?!

This is a problem in particular for elderly people who may not always have the necessary expertise to properly address security-related issues.

What it does

AugmentedOma helps (not just) elderly people to deal with IoT security in a fun, easy-to-use, and interactive way. The app runs security scans to check all devices in the household for potential security risks and vulnerabilities. But instead of displaying them in the usual cryptic way, we use Augmented Reality to almost literally take elderly people by the hand and show them around their own house and provide step by step instructions on how to fix a given problem.

This does not only empower elderly people to participate in the opportunities of the digital age. It also makes them feel safe in your own house again.

How I built it

AugmentedOma is a mobile web application that, once connected to the home wifi, automatically performs security checks (using port scanners, and public APIs). If a potentially vulnerable IoT device is found, it guides the user to its location and helps to easily find it by using object detection that runs on top of the user's smartphone camera. It then displays step by step instructions on how to fix the issue by using overlays on the smartphone camera in an Augmented Reality fashion. If the user gets stuck anyway, the user can always ask a relative for help directly from within the application. Lastly, we re-run the initial security check to see if the vulnerability has been fixed.

Challenges I ran into

  • Finding an accurate way to assess whether or not a device is considered vulnerable
  • Creating an engaging user-experience that addresses an elderly population

Accomplishments that I'm proud of

  • The quality of the object detection
  • The user experience

What I learned

  • The importance of security when it comes to IoT
  • How it can be fun to design an application for a target group you're not (yet) a part of (elderly people)

What's next for AugmentedOma

  • Improving the accuracy of the security scanning and vulnerability detection
  • Coming up with more use cases and step by step solutions for potential fixes
  • Improving the accuracy of the object
Share this project: