tilted

• 

  tilted group photo

There is more disruptive to a good jam sesh than having to pause for a page flip or, God forbid, a page scroll. Many performers have suffered from this annoyance and have resorted to hiring page turners and expensive mechanical contraptions. Surprisingly, there haven't been many accessible ways to flip a page electronically. The apps that we found were from 2-6 years ago and were not in the App Store, and another open-source app did not support glasses-wearers and the code was unavailable. We wanted to make the process simple.

We made use of PoseNet, a machine learning model developed by Google to track different parts of the human body. We used this to track the angle of our head across the frontal plane and use that to automatically turn pages on a pdf. We learned how to handle pdf rendering in Python and Tkinter.

To build our software, we split into two teams, one focusing on head tilt detection and the other on PDF rendering. Then we came together to combine our modules and fine-tune the GUI.

As we brought each part together, we refactored our code to be more object-oriented and redesigned our modules to fit together more intuitively. This was the most difficult part, as we took pains to structure this code as modularly as possible to support future development. We have high ambitions for extending the features and platforms of this project. Currently this application only works on desktop devices, and we want to see it on tablet products. In the future, we might even see communication across devices for purposes such as synchronized page turning in orchestras.

Built With

• pillow
• posenet
• pymupdf
• python
• tensorflow
• tkinter