Memento

For severe cognitive impairments, remembering where you last put things is impossible. Memento is an edge-computed vision helper tool that reduces caregiver fatigue by automatically tracking items.

Comment

Inspiration

Recent CDC data suggests that at least 64 million Americans have suffered a Traumatic Brain Injury (TBI) in their lifetime. For survivors dealing with anterograde amnesia—or individuals in the early stages of Alzheimer's—the brain loses the ability to encode new short-term memories. They can set their wallet on a table, turn around, and forget where it is moments later.

"Bluetooth trackers fail this demographic completely because they require the user to actively remember to attach them or log their items in an app. The technology defeats its own purpose."

This overlooked group doesn't need another generic tracker; they need a digital memory prosthetic.

What it does

Memento is an ambient room awareness system. It passively watches trusted drop zones in a room (like an entryway table or nightstand) and silently remembers where important objects—like keys, wallets, glasses, and phones—are left.

When the user needs to find an item, they simply open the high-contrast Memento iOS app and tap: "Where are my keys?"

The app queries the local network and uses cognitively-tuned text-to-speech and haptics to tell them exactly where the item was last seen. No tags, no logging, and no behavioral changes required.

How we built it

Because we are putting a camera in someone's home, privacy was our top priority. We built a Zero-Cloud Architecture.

  • The Vision Edge Worker: A local Python script uses OpenCV to connect to a webcam and run a YOLOv8 object detection model. Video frames are processed entirely on the edge and discarded immediately.
  • The Memory Bank: The vision worker extracts only the metadata (the object label and its room coordinates) and writes it to a local SQLite database (memento.db).
  • The Interfaces: We serve this data via a local FastAPI endpoint. Users can view their items via a Next.js web dashboard (secured by Supabase) or our custom-built SwiftUI iOS app.
  • Cognitive Accessibility: The SwiftUI app was designed with large touch targets, a dark-mode theme with high-contrast mint green accents, and a customized text-to-speech synthesizer (slowed to a 0.45 rate with a 1.1 pitch multiplier) so users have time to process the auditory feedback.

Challenges we ran into

  • Balancing Performance and Hardware: Running continuous YOLOv8 inference locally can heavily tax standard hardware. We had to implement frame-skipping and resolution tuning (down to 640px) to ensure the system could run smoothly on edge devices without overheating or lagging.
  • Camera Calibration: Mapping 2D pixel coordinates to real-world "trusted drop zones" required us to build a custom calibration script so users can easily draw bounding boxes over areas like a "desk" or "nightstand".
  • Designing for Cognitive Load: We iterated heavily on the iOS app to remove all unnecessary UI elements. We had to learn that accessibility isn't just about screen readers; it's about reducing cognitive friction.

Accomplishments that we're proud of

  • Successfully building an AI-powered system that requires zero cloud processing for its core computer vision tasks, guaranteeing user privacy.
  • Creating a truly frictionless accessibility tool. The user literally does nothing differently in their daily life, yet gains immense independence.
  • Bridging Python/OpenCV ML pipelines with modern frontend frameworks (Next.js) and native mobile SDKs (SwiftUI) seamlessly.

What we learned

We learned that AI and computer vision don't have to be deployed in massive cloud data centers to be useful. Edge AI is incredibly powerful for building privacy-preserving, local-first applications. We also learned how to read, interpret, and design for specific cognitive accessibility guidelines rather than just standard visual accessibility.

What's next for Memento

  1. Custom Weights: Training custom YOLO weights specifically optimized for varied shapes of keys and wallets, which can be tricky for standard models.
  2. Multi-Camera Support: Allowing the edge worker to stitch together multiple camera feeds to cover an entire house.
  3. Voice Activation: Integrating local wake-word detection so users can just speak to the room ("Memento, where is my wallet?") without needing to open the app.

Built With

Share this project:

Updates