In a 2012 study from the National Health and Nutrition Examination Surveys, 70% of computer workers report head and neck pain from working at a computer all day.

In June 2020, a Stanford study recorded that in the new “work-from-home-economy,” 42% of the American workforce worked at home full-time. Several companies are developing plans for more work-at-home options for the future. As of September 2020, Education Week's school reopening database reports that more than eight million students will be learning remotely in the new school year. The new reality for millions of Americans and people across the world is remote work and learning.

The problem regarding bad posture will only get worse as more people work and learn from home.

As students who study and code for hours on the computer, we understand and face the consequences of bad posture every day. We believe SlouchAI can be a solution to the bad posture epidemic.

What it does

SlouchAI uses your computer webcam to monitor your posture and notify you if our image classifier labels your position as slouching. Letting you know to stay in an ergonomic workflow and avoid chronic back pain.

How we built it

SlouchAI utilizes Google Cloud’s AutoML Vision as the image classifier to whether a user was slouching or sitting up. 200 photos of the developers and their willing family members as training data to build a general slouch classifier. Our program takes a photo every 20 seconds from the user's webcam, removing the background using python’s open-cv library to simplify the image, then sent it to our classifier to detect slouching, which would then notify the user to correct their posture.

Challenges we ran into

Throughout the development process of the project, the team deliberated on the following topics: (1) Video AI API vs. Vision API, (2) Generalized Classifier vs. Custom Classifier, and (3) Guidelines for Images (Standardizing Training Data).

Originally, our objective was to have the camera continuously monitor the user's posture by recording videos. Our plan was to use Google Cloud's Video AI API. After discussing with a Google Cloud mentor, we realized that Video AI may not be built for the capabilities we were seeking. We then decided to capture images and use Google Cloud's Vision API (specifically the AutoML API). In the end, capturing images was much easier and lighter than or an initial plan to record prolong videos. The data collection process and the training process was greatly expedited from switching over to using the Vision API.

In the beginning, our team imagined that a custom classifier would be developed for our application. We were concerned that if we chose a general classifier we would not have enough data to prevent overfitting to the teammates involved in the project. However, after consulting with the Google Cloud mentor we wanted to verify the feasibility of our project first. Additionally, the user experience would be more optimized by using a generalized classifier.

The last challenge our team faced was standardizing the training data (the images). We had to come to a consensus on the definition of Slouching. The following are the guidelines we developed to collect the 'Novel Slouch Image Dataset':

  • 30 photos per person -- 10 pictures per slouching angle --- 5 slouching, 5 not slouching -- For slouching and non-slouching slightly shift each image or change hand placement
  • Definition of Slouch: Exaggerate neck + back slouch
  • 3 Angles: Tilt slight left, tilt right, and straight on
  • Virtual background: plain color (Green)
  • Apply virtual background to ‘clean image’:

Accomplishments that we are proud of

Machine learning is at the bleeding edge of technology right now, and getting hands-on experience with applying this powerful tool to help our day-to-day lives is empowering. We will continue to learn more about this deep field of knowledge and contribute to the democratization of AI.

What we learned

We have never used a cloud-based AI platform. So learning how to connect to a cloud API and produce a good training dataset was the majority of the learning experience. We learned that having a clear and thought out design is crucial before getting into the details of the implementation.

After consulting with a Google Cloud mentor, we found that we would have gone down a dead-end if he didn’t correct our direction.

What's next for SlouchAI

Functionalities the SlouchAI team would love to have is prompting the new user to input some calibration images when they first initialize the program. New users would be directed to enter a slouched position and a non-slouched position. These photos will further improve the model and well as add a customization feature.

Built With

+ 3 more
Share this project: