chiroprac-tech

Inspiration

Sitting is one of the most common actions we take daily. It is now not uncommon for workers to find themselves sitting upwards of 6+ hours every day for 5 days a week. Therefore, we believe that sitting needs to be taken seriously. Sitting incorrectly for prolonged periods of time leads to pain, loss of mobility, and even physical deformation if not corrected. This is the goal of chiroprac-tech, we are helping people raise themselves up __ before __ they sell themselves short

What it does

Chiroprac-tech uses a Raspberry Pi camera and pose detection to watch your posture while you work. When you slouch or lean forward for more than a few seconds, an on-screen warning appears, and an audio reminder plays, prompting you to correct your posture. The point data is stored in a database where we take your weekly posture habits and present you with insights into your posture over the past week.

How we built it

Hardware Hardware-wise, we used a Raspberry Pi 4B with the provided Logitech USB camera. Using the Pi, we have the camera start recording video that was then streamed to a more powerful machine (our laptops), where we could then start to analyze and observe the sitting pattern of the person in front of the camera.

Challenges we ran into

Hardware Getting the Pi to boot onto a monitor proved to be difficult, and we ultimately had to flash an older legancy 32 bit version of the OS. Accounting for the rather high memory footprint of video was causing the Pi to crash. To mitigate this, we went with a greyscale picture being taken every 2 seconds. We also made use of the Zram library in Liniux which allowed us to get more out of Ram by reducing the time spent writing/reading data to and from the swap space (the micro SD card)

Video Pivot

We thought this was too slow and produced too jittery a movement, which proved too hard to process. We iterated to using near-real-time videos. This is the rolling video idea that had the camera start and run for say 15 seconds. From the time start we wrote data to a file on the Pi. At time 5, we would start writing to a second file and still to the first. At time 10, we would open a 3rd file and start writing to all three. This was all done in an attempt to get over the time skip of having to stop recording and sending this data into MongoDB.

Live Video

This video record -> upload -> download + process pipeline was just not the best way to do this. We remembered what we had heard of people hosting media servers on Raspberry Pi, so we started to tinker with the idea here. The result was the final product of a live-streamed video sent from the Pi onto our local machines. This removed the complexity of working with MongoDB and sped up the process, making it truly real-time.

Software

A small Python program runs on the Pi. It reads video from the camera, turns each frame into a JPEG, and sends them one after another over the network. Another Python program runs simultaneously that fetches the live stream from the PI, runs an AI model that finds the body's landmarks: shoulders, ears, and hips. Then it calculates the angles from those landmarks (neck tilt and how much you are leaning forward) to draw overlays indicating your posture rating. It finally sends those annotated frames to a webpage. The Next.JS app serves as a webpage that displays the live video with overlays while also providing data for live metrics and metrics for weekly posture overviews.

Accomplishments that we're proud of

Hardware

Things we are proud of in the realm of hardware are that we have produced a system that does not crash, given the data it is working with. The pivot to real-time was a major point of achievement for the team, as this really felt like our project was moving into a modern product

What we learned

Computer vision was a major learning point for this project. Stepping back into the Stone Age with memory-strapped systems was a fun learning curve that has brought more appreciation and thanks to the hardware that we use every day