We had some background in developing apps for the windows store using Xamarin, so we decided to make something on that framework.
What it does
Ideally, the app would allow you to point your phone a flat surface that you can pretend is a keyboard. By then pretending to type on that surface you can use your phone as an external keyboard or just type into your phone.
How we built it
Our application is built on Xamarin and uses OpenCV to process the camera feed.
Challenges we ran into
Getting the raw camera feed inside of Xamarin was harder than we were expecting it to, and the Python code we had working for OpenCV image processing proved too difficult to move over to C# on Xamarin.
Accomplishments that we're proud of
We wrote a mobile app from scratch in Xamarin and developed a custom computer vision algorithm. Capturing the camera and manipulating it on-screen was challenging but we ultimately came up with a solution. On the side we even wrote an FPGA accelerator for later versions of this project.
What we learned
Computer vision is hard and making it work in real time is difficult, especially on a mobile device. Manipulating raw image data from the phone camera is poorly documented.
What's next for Its not the hack, we are!
We are the hack!