Inspiration
We were tired of losing keys, wallets, phones, and AirPods. Every one of us has torn apart a bedroom for something that was sitting in plain sight. We wanted "Ctrl-F for the real world".
What it does
Press the button on our Raspberry Pi camera, or visit ctrl.photo and ask out loud for what you've lost. The Pi takes a photo, a custom-trained detection model searches it, and a witty steampunk narrator tells you where your item is.
How we built it
- Hardware: a Raspberry Pi with a Hermes camera and a button to trigger a capture.
- Web: ctrl.photo, where you can talk to the narrator.
- Vision: a custom-trained object detection model that looks for our target items in each photo.
- Voice: Gemini writes the narration, and ElevenLabs speaks it in character.
Challenges
Training the model was the hardest part. We needed far more data than we expected, and almost none of the available datasets matched our use case. Most public data shows clean, well-lit objects. We needed to find small items in messy places like our bedrooms, with clutter, odd angles, and poor lighting.
What we learned
A good model depends on good data. Collecting data that matches the real environment mattered more than anything else we tuned.
What's next
More item classes, and better accuracy in cluttered rooms.
Built With
- elevenlabs
- gemini
- machine-learning
- visionproject
Log in or sign up for Devpost to join the conversation.