_PixelPose

Team RAJE Members
Play a Wii Inspired Game
And pose for the camera!!
PixelPose Main Page
Draw a Pose in the Pose Maker
Match the Pose in Live Pose Detection
Shrimp Detection Showcase

Inspiration

As programmers, we are very familiar with bad posture habits, as are many other gamers and computer enthusiasts. In one 2022 study, 70.5% of respondents complained of musculoskeletal discomfort related to "sitting at a table, alongside a prone position whilst using a mobile phone, as well as a latent or floor-sitting posture when using a laptop, whether at a table or not". Additionally, bad posture can lead to chronic pain and other medical problems. Aesthetically, we were also inspired by WiiPlay's Pose Mii, Hole in the Wall, Super Mario Maker and other physical pose-related games for our concept.

What it does

_PixelPose has built-in "shrimp"-detection, using a camera feed and MoveNet's body detection with custom polygon reconstruction, to detect when a user's posture might be bad, and encourages them to play our Strike a Pose game! To play, simply load up a user-submitted pose, and try to make the pose within the timer. Users can also submit their own poses and share them with friends and other users, so long as the pose is actually possible to recreate with your body.

How we built it

We started by building out our frontend, UI, and pixel pose drawer, then moved on to the backend and authentication, where we spent most of our time. We then attempted to host the web app using Vercel, but ultimately decided to focus on the implementation and visuals instead. Finally, we made a short video showcasing the game and its features.

Challenges we ran into

Because none of our team members have experience creating a fully-integrated backend, we used LLMs to learn how to make it. It was challenging getting the backend and frontend to communicate, especially because of the number of different ports, files, and routes involved. We overcame these challenges by watching tutorials, reading documentation, and using whiteboards to jot down our process.

Accomplishments that we're proud of

Red implemented the login and authentication processes for account creation. Amanda created the "shrimp" detection algorithm to automatically sense when a user needs to improve their posture. Jamie created a pixel canvas that users can draw poses onto. Eric used MoveNet, a TensorFlow ML model focused on body detection, to create a live feed body reconstruction that translated to a binary 2D array.

What we learned

We learned an incredible amount about using these different frameworks for our web app. We worked concurrently using Github, and because of our experience working together, we were able to minimize merge conflicts. However, we learned the importance of establishing a hosting strategy and backend infrastructure early as part of planning out a web app, which we didn't realize until we had coded a lot of the game logic and user interface.

What's next for _PixelPose

With more time, we would like to fully integrate the backend so that users can seamlessly use the web app as a website with more fleshed-out features like a friends list and a grid of user-submitted poses to choose from. We also want to debug our game a bit more, because the logic is conceptually sound and fun, but the implementation could use refinement. Finally, we want to nail down hosting the website somewhere on the Internet.