ErgoBot

ErgoBot is a posture-mimicking desk companion with a real-time web application to monitor sitting posture and visualize ergonomic trends, aiming to help office workers build better posture habits.

Comment

Inspiration

Initially, we approached the hackathon by looking for a problem that could combine robotics, machine learning, and public health that is relevant within our demographic. Looking around at people staring at their computers led one of us to think about their atrocious posture as they stared at a screen waiting for ideas. After thorough research into the field of ergonomics, we decided to approach the issue of fixing posture for the demographic most at risk of musculoskeletal disorders: office workers.

Office workers are often forced to endure sitting for prolonged periods, making posture a dangerous ergonomic issue. Awkward postures can contribute to muscle fatigue and musculoskeletal disorders with age. In 2025, software developers spent an estimated 97.1% of their workday sitting, and the National Library of Medicine published an article in 2024 that reported musculoskeletal diseases in as high as 89% of software engineers.

Poor posture eventually becomes automatic. A person may not realize that they have been slouching for several minutes until the discomfort has already developed. We wanted to make something that could identify posture changes and communicate them in a way that a busy computer user could not easily overlook.

What it does

ErgoBot is a posture-monitoring robot that uses computer vision to identify and mimic a user’s body position while sitting in a chair. When a user slouches, for example, the robot recreates that motion in its upper torso.

It is built with MediaPipe to process webcam footage to evaluate the angles of the user’s torso and head relative to a configured “good posture” position. The system connects that analysis to both a servo-driven robotic body and a web interface to make posture monitoring more upfront to office and tech workers. Throughout a session, the interface uses real clinical-based assessment techniques---RULA (Rapid Upper Limb Assessment), REBA (Rapid Entire Body Assessment), and the clinical Plumb Line Model---to grade and advise the user about how to improve their spinal or neck orientations. The interface also shows a graphical analysis, history, and general report of a user’s posture throughout a session.

However, a posture warning on a screen is easy to ignore, especially if the user is focused on work. We wanted to make it impossible for a user to overlook the state of their posture to encourage a correction and prevent bad postures for extended periods of time. My teammate and I designed ErgoBot as a desk companion that takes data from an algorithm using the MediaPipe library and the user’s monitor camera to observe the user’s posture and mirror it with its own body. When the user slouches or leans, the robot reflects that posture back to them.

A physical robot is the ideal solution to this problem because ergonomics itself is fundamentally physical. A conventional dashboard, like the interface attached to ErgoBot, can display scores, but what does that mean to the user? The robot can communicate posture through movement, which is more effective for a busy human user. The robot is also the foundation for interactions like physical reminders and break prompts.

How we built it

ErgoBot uses a camera and MediaPipe Pose to identify body landmarks and estimate four posture-related measurements: head and neck position, trunk or spine alignment, shoulder symmetry, and pelvis or hip tilt. The user first calibrates the system in their normal, comfortable “good posture.” Measurements are then compared against that personal baseline rather than against a one-size-fits-all definition of correct posture, which may be ineffective for those with scoliosis or other conditions.

This personalization was important to us because bodies and natural sitting positions differ. A universal posture standard may not represent every user equally, especially users with scoliosis, previous injuries, or other physical differences. Instead, ErgoBot primarily focuses on detecting changes from the user’s own baseline and making those changes visible as they occur.

The scoring algorithm begins at 100 points and applies penalties when the user’s measured posture moves beyond the established thresholds. The penalty structure is informed by ergonomic assessment methods including RULA and REBA, with separate penalties for head, spine, shoulder, and pelvis deviations.

The score also accounts for duration. Remaining in a poor posture for a sustained period increases the fatigue penalty, while an exponential rolling average prevents the score from changing abruptly because of one noisy frame or a small movement. The interface translates the result into broader score ranges so that users can understand their status without needing to interpret every measurement themselves.

The Flask backend manages the camera worker, MediaPipe processing, posture calculations, status routes, and video stream. The frontend polls the backend for current measurements and turns them into visual feedback. The physical robot forms the final output layer by mirroring the user’s posture through servo-driven movement.

Challenges we ran into

One of the most important challenges was translating the noisy camera data into actually reliable feedback. Lighting, camera angle, partial visibility, distance from the camera, clothing, and temporary detection errors can all affect MediaPipe’s pose landmarks. Some body landmarks, particularly the ears, may not be visible even when the shoulders and hips are detected correctly. To address this, we added visibility checks so that the system would not treat every frame as equally reliable. We also used smoothing to reduce sudden changes in the measurements and calibration to make the system respond to each user’s natural posture instead of using only generic assumptions.

Another challenge was coordinating MediaPipe inference, camera capture, Flask routes, frontend polling, video streaming, and robot control at the same time. A camera worker had to process frames without blocking the web server, while the frontend needed to receive updated posture information continuously. Managing shared state, stopping and restarting the camera, and keeping the live stream responsive required careful handling of threads and locks.

Finally, we faced another challenge in presenting technically complex information without making the interface feel overwhelming. The system processes multiple anatomical measurements, but users do not need to see every implementation detail at once. We therefore designed the interface around a small number of readable outputs: live status, session grade, posture timer, anatomical deviations, analysis graphs, and practical suggestions.

Accomplishments that we're proud of

We are proud that ErgoBot connects a machine-learning-based computer-vision pipeline to a physical robot and a web application. The project is not only a posture classifier, a dashboard, or a robotic demonstration.

We are also proud of building a personalized calibration system. Instead of assuming that one posture is correct for every person, ErgoBot establishes a baseline from the user’s own comfortable position. This allows the system to focus on meaningful changes and makes the feedback more adaptable.

The scoring system, too, was a huge accomplishment for us. The score is not based on a single arbitrary posture label. It combines multiple anatomical deviations, calibrated thresholds, time spent in poor posture, and rolling averaging. This makes the feedback more expressive than simply labeling a person as “good” or “bad.” The score ranges are informed by RULA and REBA action levels, ranging from strong posture to a recommendation to take a break and stretch. It took a significant amount of research and extensive thinking on how best to apply mathematical formulas to the scoring guide.

We are also proud of the interface design. We kept the visual language minimal so that the complexity of the computer vision and scoring system would not make the product difficult to use. The dashboard gives users immediate feedback, while the Analysis section explains how posture changed over time. The physical robot provides an additional layer of feedback for users who may ignore or miss a notification on their screen.

What we learned

Through the project, we learned that a machine-learning system has to do so much more than produce predictions. It has to communicate uncertainty, recover from imperfect input and data, and present its output in a way that people can understand and act on.

We also learned that using machine learning does not mean every part of the system needs to be a machine-learning model. MediaPipe provides pretrained machine-learning-based pose detection, but our posture scoring, calibration, fatigue adjustment, and rolling average are deterministic algorithms that we designed around the detected landmarks. Keeping those separate made the system easier to explain, test, and debug.

Hardware can make software feedback more meaningful, too. A number on a screen can be easy to ignore, but a robot physically copying a user’s posture creates a stronger connection between the measurement and the user’s body.

Further from a software perspective, we learned how important state management and concurrency are in a real-time application. Camera capture, model inference, web requests, frontend updates, and robot commands all need to operate without blocking one another. We also learned that reliability is part of the user experience: the system needs to handle missing landmarks, camera interruptions, and incomplete sessions gracefully.

What's next for ErgoBot

The next step for ErgoBot is to strengthen its long-term analytics, which would allow users to see whether their posture improves over several work sessions rather than only seeing what is happening in the current moment.

We also want to measure the effectiveness of the robot’s feedback. For example, the system could record when the robot mirrors a poor posture, how long it takes the user to correct that posture, and whether the user’s score improves afterward. This would help us evaluate the robot as a closed-loop intervention rather than only as a monitoring device.

Future versions could use the robot for physical reminders and micro-break suggestions when poor posture continues for too long. The robot could raise its head, roll its shoulders, change its lighting, or display a simple message encouraging the user to stretch.

We would also want to develop a posture game connected to the web application. A user’s posture quality could control the progression or condition of their character, while friends could compete through shared scores or session results. This would make posture improvement more engaging without changing the central purpose of the project.

Additional future work includes multi-user profiles, persistent calibration histories, improved hardware movement, stronger landmark validation, and more detailed robot responses. As the system develops, we would also want to evaluate its measurements more carefully against established ergonomic practices and ensure that its feedback remains supportive rather than judgmental.

ErgoBot combines machine learning, computer vision, physical computing, backend engineering, and UI/UX around helping people notice and correct unhealthy desk postures before they become serious or irreversible habits. It is not intended to provide a medical diagnosis, but we believe its physical feedback and personalized approach could provide meaningful value for office workers, students, software developers, and anyone who spends long periods working at a desk.

Built With

Share this project:

Updates

Submission history