Beacon

Every recognized face gets a card: who, where, when. AMBER alerts in red. Today’s path is drawn on the map; click it for Street View.
Beacon dashboard: live glasses feed, lens display, people met nearby, AMBER watchlist, and a Street View of your walk.

Inspiration

Beacon was inspired by a simple question: what if smart glasses could help you remember the people you meet and where you last saw them? We wanted to explore how wearable technology could assist with memory, navigation, and locating someone during situations such as a missing-person alert—all while keeping the wearer’s hands free.

What it does

Beacon turns Meta smart glasses into a wearable visual-memory assistant. The glasses stream camera frames through an Android phone to our backend, where faces are detected and converted into embeddings. Beacon can recognize someone it has seen before, display their name or notes on the glasses, and record where and when they were encountered.

Users can name people, add notes, create watch alerts, and ask where somebody was last seen. Beacon also builds a map from the wearer’s movement and can guide them back to a person’s last known location. Full frames and face crops are stored privately so users can review and correct recognition results.

How we built it

We built the Android application in Kotlin using Meta’s Wearables Device Access Toolkit for camera streaming and glasses display output. The phone sends frames and location information to a Python FastAPI backend.

For face recognition, we use the AuraFace v1 model from Hugging Face through InsightFace and ONNX Runtime. Faces are represented as 512-dimensional embeddings.

Neon powers the data layer. Neon Postgres stores people, embeddings, sightings, locations, map nodes, and routes. The pgvector extension performs similarity searches between face embeddings, while Neon Object Storage privately stores face crops and complete sighting frames. The backend connects each stored image to its corresponding database record.

Our navigation system builds a graph from the wearer’s movement and uses A* pathfinding to guide them toward a person or location. The final result is displayed as simple, readable instructions on the glasses.

Challenges we ran into

Combining real-time camera streaming, face recognition, database writes, object storage, location tracking, and glasses output created several integration challenges. We had to control latency so processing one frame would not block incoming frames or make the display feel delayed.

Face quality also varies significantly with distance, lighting, motion blur, and viewing angle. We created quality thresholds and training sets so Beacon only saves useful face images.

Migrating from local SQLite and filesystem storage to Neon required redesigning the database layer, adding Postgres migrations, configuring pgvector, and replacing local image paths with private Object Storage keys. We also had to prevent embeddings generated by different recognition models from being mixed together.

Accomplishments that we're proud of

We created an end-to-end system that works with real smart-glasses hardware rather than only a simulated interface. Beacon can detect a face, generate an AuraFace embedding, compare it through Neon, store the sighting and its images, and return useful information to the glasses.

We are especially proud of connecting face recognition with location-aware memory and navigation. Beacon does more than identify someone—it remembers the context of the encounter and can help the wearer return to that location.

We also added practical privacy and correction features, including anonymous identities, configurable retention, manual merging and reassignment, private image storage, and a complete data-reset option.

What we learned

We learned that a machine-learning model is only one part of a useful AI product. Data quality, storage design, latency, model compatibility, privacy, and user experience are equally important.

We also learned how Neon Postgres, pgvector, and Object Storage can work together as one backend for structured records, vector similarity searches, and private media. Working with wearable hardware also taught us how important graceful failure, concise displays, and real-device testing are.

What's next for Beacon

Next, we want to deploy the backend to scalable cloud infrastructure, improve recognition accuracy through additional real-world testing, and optimize inference for lower latency. We also plan to add authentication, encryption, consent controls, and stronger automatic retention policies.

We would like to use Neon database branching for safe development and demonstrations, add more advanced location understanding, and evaluate hosted GPU inference for supporting multiple wearers. Our long-term goal is to turn Beacon into a reliable and privacy-conscious memory assistant for accessibility, safety, and everyday use.