We had noticed a trend that our postures in our lives weren’t the greatest and that our parents and colleagues are constantly commenting on it. We had realized that this trend was heavily influenced by posture while sitting down, especially when using electronics. With technology becoming a ubiquitous utility in our lives, our poor posture needs to be corrected before it becomes permanent and harmful to our health. That’s why we created PoseRight, a tool to monitor and encourage proper posture while sitting at a computer.
What it does
PoseRight uses a camera to analyze the user's posture using the pre-trained PoseNet model. from Tensorflow. The camera is connected to a Google Coral development board, which analyzes the data from the camera to give a rating on their posture. The resulting data is sent to a server and database where it’s later retrieved by a Google Chrome extension and the website. If the extension starts to detect the user’s position is worsening, the screen will blur the tabs that the user is using. It unblurs if their posture improves. With the Coral development board, we are able to process all the data client side. This serves two main purposes, the first is that it classifies in approximately 2 milliseconds since network latency is eliminated. This speed is important when working with real time data. Secondly, it eliminates any privacy concerns as the live video feed is never stored or exposed to the internet, ensuring that you are in control of your own data. In the end, the data stored in the database is graphed in two charts, allowing the user to see their posture over time.
Challenges we ran into
There was a more accurate network architecture (ResNet50) available for Tensorflow.js, but we were unable to convert the model to conform to the Edge TPU standard. At first, we were unsure of how to interface between the three points in the project; however, in the end, we opted to use a solution where the extension and Coral board used the server as an intermediary to communicate with each other by pulling and pushing data. To link everything together, we used a Docker container, but many of our dependencies were not available on Alpine Linux which forced us to find alternatives.
Accomplishments that we're proud of
We are proud of creating a working final product that was both consistent and helpful. We were able to integrate 3 aspects of our project together (Chrome extension, Server, Coral Board). We were also proud of the way that we processed the data because it was efficient and reliable for many different users.
What we learned
One major thing that we learned is being able to create and implement a Google Chrome extension that was constantly updating in the background. We learned how to work with Tensorflow Lite models as well as the PoseNet model in particular.
Posture is important!
What's next for PoseRight
With additional time, we would be able to train our own model which would specialize in the upper body while sitting. In future iterations, we would like to implement a system-wide blur feature, and physical reminders such as lights or vibration motors. We would also create a way to manually override the blur in extenuating circumstances.