Every day, millions of underrepresented people across the globe fail to speak up about issues that affect them. This may be due to fear of public speaking or a lack of proper presentation etiquette. No matter the problem, the effect remains the same: a large number of thoughtful, detailed ideas will never be shared with the world.

While reading research articles to create a solution, our team came across an article that proved that fear of public speaking was correlated to lack of experience and proper presentation etiquette. For many people, their fear of public speaking stems from a lack of presentation experience. Because of this, these individuals also lack proper presentation etiquette, which snowballs into even more fear. This vicious cycle of fear and inexperience acts as an obstacle preventing people from sharing their ideas with the world.

Thus, our team developed Posture Buddy, a user-friendly application that uses AI to provide users with experience in public speaking through algorithms that detect common forms of bad posture and provide custom reports on how to fix it.

What it does

Posture Buddy is a progressive web application that detects bad posture and helps aspiring public speakers improve their presentation etiquette.

Posture Buddy utilizes a two-step detector-tracker ML pipeline to identify major signs of bad presentation posture and displays all this information in a user-friendly interface. Alongside this, Posture Buddy provides users with custom tips and scoring for things like fidgeting, leaning, focus, and much more. In addition, users receive personalized tips for areas they need the most help on. With this tool, presenters can now focus on creating the best proposal without having to worry about looking good on stage.

We pulled data from thousands of hours worth of presentation data to train our model so that it could accurately discern between good and bad presentation posture. We took data from TED talks, live conference presentations, and a few college lectures to ensure that we could tailor to our users' unique needs. On top of that, we compiled data from many public speaking websites to write detailed tips on how to correct bad posture so that our users can quickly correct their bad habits.

All processing is done in real-time using either an inbuilt laptop webcam or an external one for near-instantaneous results. This means that our users won't have to sit and look at a loading screen for long periods of time just to see how well they did.

How we built it

After around an hour of planning and wireframing, we ultimately divided our tasks into two main groups. Nathan would work mostly on the machine learning model and backend, while Mithil works mainly on the user interface and design aspects of the project.

We chose to use Python as our backend language of choice because of its applications in machine learning and computer vision, making it the prime language to use. Posture Buddy uses Flask due to it being Python’s signature lightweight backend web development library. This means that our users get a fast and streamlined experience while using Posture Buddy. In addition, we used OpenCV to grayscale and downsize the video feed so that our model could more easily detect bad posture.

We also used a python library called MediaPipe to help train our model and get it accurate enough to our standards. With the help of MediaPipe, our model is able to accurately find and detect key landmarks on the human body that can be later used to determine signs of bad posture.

Finally, we developed algorithms to interpret the data from our model and to represent it in an easy-to-understand format for our end users. However, we included a stats for nerds section for our advanced users so that they can get more detailed statistics about their presentation

Challenges we ran into

Initially, we wanted our users to input a pre-recorded video of themselves presenting so that we could perform our analysis on a more powerful server PC so that users would be able to quickly get results. However, after some initial testing, we found that making users create a new video every time they wanted to present was a very tedious process, which would result in negative feedback. To fix this, we chose to do real-time analysis instead. In order to accomplish this, we implemented JPEG video streaming, where the browser repeatedly updates a single image many times per second to give the illusion that a live video is playing. This was done so that Posture Buddy can run without slowing down the application or the browser as a whole. Each JPEG image is processed by our model and then returned to the browser so that the user can see it working in real-time.

After some optimizations to both the model and the quality of the input image, we were able to achieve a stable 25 frames per second on laptops without a dedicated GPU, meaning that more powerful computers would be able to handle Posture Buddy with ease.

Accomplishments that we're proud of

We are incredibly proud of ourselves for being able to create such a project that would be able to change the course of many people's lives across the world while still maintaining uniqueness. We are in awe as to how we were able to accomplish so much in such little time and how nicely the finished product came together.

What we learned

Our team learned a lot about machine learning and web development throughout the duration of this project. We learned about libraries like OpenCV and MediaPipe and discovered some interesting new functionalities in libraries we already use like Flask. We learned how we could use computer vision to help solve issues in our community and how we could implement our solutions into interactive applications.

What's next for Posture Buddy

We want to develop a mobile app for Posture Buddy so that anyone can take this powerful tool with them on the go. In addition, we also want to add more detection algorithms so that more signs of bad posture can be detected, like slouching. We would also like to provide a free public API so that anyone can integrate Posture Buddy into their own personal projects. Finally, we want to expand the Posture Buddy app to other posture-related fields like physical therapy and sports so that more people can benefit from our software.

Built With

Share this project: