Always Watching

Introduction

Inspired by the many techniques taught in my cybersecurity class, I have been inspired to create an art installation demonstrating the many, many ways in which technology can be used to track you, simply by just being near another device.

What it does

My hack utilizes machine learning and the YOLOv3 object detection model to identify objects in real time, along with my macbooks built-in bluetooth controller to detect all of the bluetooth devices nearby. It displays all of this information in an overwhelming, encapsulating way, representing the way that people cannot escape from being in a constant surveillance-state with modern technology.

How I built it

This project uses Python, along with the ImageAI and simpleyble Python libraries to get and display the data.

Challenges I ran into

Initially, the goal of this project was much more broad, including detecting nearby Wi-Fi networks, retrieving weather data from the local IP address, and using the microphone to do live speech-to-text. Despite getting very close with all of these goals, the different time requirements for each method conflicts with another, making it difficult to implement outside of a standalone environment. on top of that, all of the Wi-Fi libraries that i found were either incompatible with macOS, or required a level of development that was unrealistic to achieve in the given time limitations.

Accomplishments that I'm proud of

The original ImageAI library that I was using was unoptimized for macOS, only providing GPU acceleration for Nvidia GPUs. I dug through the dependencies that it was using and found that it uses PyTorch, which does have macOS Metal Graphics API support, and was able to port all of the binds to Metal due to the universal nature of PyTorch.

What I learned

It's important to plan ahead for any issues during your programming, and to use a development environment that gives the best chance of everything working. Additionally, this project would have greatly benefited from additional hardware such as IR sensors, Radio receivers, etc, along with a more powerful processor.

What's next for Always Watching

The next version will contain proper support for detecting Wi-Fi networks and Speech-to-Text support, along with a redesigned GUI. For future iterations, I hope to bring multiple external devices to gather data through more specialized sensors and prioritize the object recognition on my main device. Eventually, I hope for Always Watching to be a series permanent installations.

Built With

• bluetooth
• machine-learning
• object-recognition
• python
• yolov3