Imposture

• 

  Front picture of our device

Inspiration

Watching the hackers around us work, we realized something. We're all guilty of it: slouching over our laptops for hours, picking up our phones mid-grind, losing focus every few minutes. We wanted something that would actually call us out, but in a fun and cute way. Taking inspiration from pet companions such as tamagotchis, we considered flipping the roles, having this Imposture take care of us instead. In addition, we wanted to involve the hackathon's theme of Love by encouraging self love through self-care, another important aspect of life that many people in the growingly busy digital society begin to forget.

What it does

Imposture is a little desk pet that sits next to your laptop and watches how you work. It uses a webcam to track your posture and catch you reaching for your phone. Slouch too long and Imposture gets sad. Pick up your phone and it gets mad. The screen shows an angry face and buzzes at you until you put it down. Stay focused and Imposture is happy. You can even talk to it ("Hey Imposture, start a Pomodoro Session"), earn XP for good behavior, and check your stats on a dashboard in your browser.

How we built it

The brains run on an Arduino UNO R3. We use MediaPipe to track body landmarks from the webcam and score posture based on spine angle, head tilt, and shoulder alignment. And, an EfficientDet model handles phone detection. The Arduino drives a little LCD screen that shows kaomoji faces, a buzzer, and a vibration motor for feedback. For voice, we chained together Picovoice for wake word detection, Deepgram for transcription, ChatGPT for understanding what you said, and ElevenLabs to talk back. The dashboard is a FastAPI app with WebSocket streaming so you can see your posture score and webcam feed live.

Challenges we ran into

Our original idea was to use a Raspberry Pi connected to the Arduino, but we couldn't get it to connect to the wifi. We tried using ethernet, but it seemed like the chip was broken, because it would never boot. Then, we pivoted to using just the Arduino, losing a lot of the functionality we had planned for. On top of that, we had never used one before, so the process of learning to wire it took ages. We also had a bug where phone alerts and posture alerts were fighting over the LCD, overwriting each other every second, and it was nearly impossible to debug. On the Arduino side, we were running out of memory for the actual firmware, and constantly had to simplify and compress our code to fit the space boundaries.

Accomplishments that we're proud of

We are proud of learning how to wire Arduinos and Raspberry Pis, as well as designing and building 3D printed casing. We also learned how to apply MediaPipe and OpenCV to real world actions and subjects. Finally, we learned how to connect STT and TTS workflows with Deepgram and ElevenLabs, learning new frameworks that unlock endless possibilities for future projects!

What we learned

The biggest thing was that calibration matters way more than we expected. Our posture scoring didn't really work until we added a step where you sit up straight and it learns what "good" looks like for you specifically. We also learned that physical feedback, like an actual buzzer going off on your desk, is way more motivating and stimulating than any screen notification. Finally, we also learned about how important precise measurements are for physical products. Dealing with the variability in the 3D printed objects forced us to redesign our items multiple times and learn the nuances of printing.

What's next for Imposture

We want to refine the design of the cases and the assembly. We also want to take more variable user inputs through voice commands, so the user can talk to it casually (such as a therapist). We want to implement more features like lights, buzzers near the bottom of the feet so we can expand the range of cues the user can be alerted through, such as vibrations.

Built With

• adafruit
• arduino
• deepgram
• elevenlabs
• fastapi
• jinja
• liquidcrystal
• mediapipe
• opencv
• picovoice
• raspberry-pi