PoseMon

MoveNet labeling
cross legged warning
slouching warning
tensorflow Model

Inspiration💭

I was inspired to create this app because I personally struggled with maintaining a healthy sitting posture for long periods of time. I often experienced discomfort and realized that poor posture was negatively affecting my overall well-being. I wanted to find a solution not only for myself but also for others who might be facing similar issues.

Additionally, I have always been passionate about health and wellness. I saw an opportunity to combine my interest in technology with my desire to promote better posture habits. The advancements in AI and computer vision technology, particularly MoveNet, intrigued me. I recognized its potential to accurately analyze and evaluate sitting positions in real-time.

My ultimate motivation was to help people become more aware of their sitting habits and provide them with a tool that could guide them towards healthier choices. I believe that by incorporating AI into a mobile app, we can empower individuals to take control of their posture and improve their overall well-being.

Seeing the positive impact this app could have on people's lives and the potential for it to be an educational tool further motivated me. I wanted to create an app that not only detects unhealthy sitting positions but also educates users about the importance of proper posture and its long-term effects on their health.

What it does🙊

PoseMon app is designed to assess and evaluate the user's sitting posture using AI technology. By utilizing the MoveNet Thunder model for human pose estimation and a classification network for posture classification, the app can analyze the user's sitting position in real-time.

The primary purpose of the app is to determine whether the user's current sitting posture is considered healthy or not. It categorizes the sitting posture into three states: "Standard Sitting Posture," "Crossed Legs," and "Forward Head and Hunched Back." The app provides feedback to the user based on the assessment, indicating whether their sitting posture aligns with the recommended standard or if adjustments are needed to maintain a healthier posture.

In summary, the PoseMon app functions as a real-time posture assessment tool, leveraging AI algorithms to identify and classify the user's sitting posture, with the aim of promoting healthier sitting habits and reducing the risk of musculoskeletal issues associated with poor posture.

How we built it💪

PoseMon app utilizes two neural network models to perform its functions. The first model is MoveNet Thunder, a lightweight human pose estimation model developed by Google. It accurately estimates human body poses and has two versions: Thunder and Lightning. In this app, the Thunder version of MoveNet is used because of its higher accuracy, although it runs at a slower speed. The Thunder version comes in two data formats: float16 and int8. The float16 model can run on general-purpose GPUs, while the int8 model can run on both general-purpose GPUs and the Hexagon DSP (Digital Signal Processor). Running on the Hexagon processor provides faster and more power-efficient AI execution, making it the recommended choice for mobile deployment.

In addition to MoveNet Thunder, the app utilizes a simple fully connected network to classify the pose information (coordinates of 17 body keypoints) outputted by MoveNet. This classification network categorizes the person in the frame into one of three states: "Standard Sitting Posture," "Crossed Legs," or "Forward Head and Hunched Back." The details and training process of this classification network are available in TensorFlow Lite's Jupyter Notebook tutorial or the modified and annotated version provided in this project. The app collected approximately 300 photos for each posture category as the training set (a total of 876 photos) and about 30 photos for each category as the test set (a total of 74 photos).

Challenges we ran into😵

Data Collection: Collecting a diverse and representative dataset of sitting postures for training the classification network can be challenging. Gathering a sufficient number of labeled images for each posture category and ensuring the dataset's quality and balance require time and effort.
Model Selection and Optimization: Choosing the appropriate AI models, such as MoveNet Thunder for pose estimation and a classification network for posture classification, can be a challenge. Evaluating different models, considering their performance, accuracy, and resource requirements, is crucial to ensure reliable results within the limitations of a mobile app.
Real-time Performance: Achieving real-time performance for pose estimation and posture classification on mobile devices might be challenging due to computational limitations. Optimizing the models and algorithms to run efficiently on mobile platforms, leveraging hardware acceleration when available, can be a complex task.
Accuracy and Robustness: Ensuring accurate and robust posture classification in various real-world scenarios can be challenging. The app should be capable of handling different lighting conditions, backgrounds, clothing variations, and individuals with different body shapes and sizes. Fine-tuning and improving the models to generalize well and handle such variations is a continuous process.

Accomplishments that we're proud of😁

Accurate Pose Estimation: Developing an AI-powered pose estimation system using MoveNet Thunder that accurately detects and tracks human body poses in real-time. Achieving reliable and precise pose estimation is a significant accomplishment as it forms the foundation for further analysis and classification.
Posture Classification: Training a classification network to identify and classify different sitting postures based on the detected body poses. Successfully implementing the classification model and achieving satisfactory accuracy in distinguishing between "standard sitting," "cross-legged," and "hunched forward" postures demonstrates a significant milestone.

What we learned📖

AI Model Integration: You gained experience in integrating pre-trained AI models, such as MoveNet Thunder, into your mobile app. This involved understanding model compatibility, optimizing performance, and leveraging the capabilities of AI models for real-time pose estimation.
Mobile App Development: Developing a mobile app with AI capabilities provided an opportunity to enhance your skills in mobile app development frameworks and technologies. You learned about platform-specific requirements, user interface design, performance optimization, and handling mobile device resources efficiently.
Dataset Collection and Annotation: Collecting a diverse dataset of sitting postures and annotating it for training the classification model involved careful planning and organization. You learned about the importance of data quality, representative samples, and data preprocessing techniques to improve model performance.
Training and Fine-tuning Models: You gained hands-on experience in training and fine-tuning AI models using your annotated dataset. This involved experimenting with different architectures, hyperparameters, and training strategies to achieve optimal performance for posture classification.

What's next for PoseMon😎

Real-Time Feedback: Enhance the app to provide real-time feedback to users about their sitting posture. This can involve integrating visual cues or audio alerts to guide users in adjusting their posture for better ergonomics.
Customization and Personalization: Allow users to customize the app based on their specific needs and preferences. This can include adjustable sensitivity levels, personalized recommendations for posture correction, or tailored exercise routines to address individual posture issues.
Posture Tracking and Progress Monitoring: Implement a tracking feature that records and monitors a user's sitting posture over time. Provide visualizations or progress reports to help users track their improvements and encourage them to maintain healthy posture habits.
Gamification Elements: Gamify the posture improvement process by incorporating challenges, achievements, or rewards. This can motivate users to actively engage with the app and develop healthy posture habits through a fun and interactive experience.
Integration with Wearable Devices: Explore the possibility of integrating the app with wearable devices, such as smartwatches or posture-correcting wearables. This can provide additional data inputs, improve accuracy in posture tracking, and enable users to receive posture-related notifications directly on their wearables.
Cloud Sync and Multi-Device Support: Enable users to sync their posture data across multiple devices through cloud storage. This ensures seamless access to their posture tracking history and personalized settings from any device.