I've always been frustrated by the sheer amount of remotes and controllers that clutter our homes, and I have always wondered if there was a better alternative.

What it does

It uses the phone's camera to determine the phone's position and orientation, so it can dynamically change the interface depending on what it's pointing at. This allows the phone to be used as a universal remote, with no extra hardware needed.

How I built it

I first created an image map of the floor, and then used the phone camera to take captures of small sections of the floor, and then used OpenCV to find matches between the phone captures and the floor map to determine where the phone was relative to the floor plan. Then I used a web app to display a remote interface that changed depending on where the phone was pointing. I also used chrome extensions to control different websites from the phone's remote interface.

Challenges I ran into

OpenCV turned out to be pretty challenging, because the feature matching is not perfect, so I had to manually tune many of the variables, like the threshold for matches to be factored into the feature matching. In addition, I was using a node.js wrapper on openCV, but it turned out to have a few issues and very little documentation.

Accomplishments that I'm proud of

I've always had very ambitious ideas, and I never end up finishing them. But 10 hackathons later, I believe this is my first time finishing a project to a satisfying conclusion.

What I learned

Over the years, I've learned how to prioritize and manage different aspects of a project to be able to push out a prototype in short time.

What's next for Hexus

Kickstarter? :D

Built With

Share this project: