Altitude

A multimodal AI system that sees, listens, and builds a living social memory in real time.

Comment

Inspiration

Our daily social interactions are filled with many important details - names, faces, shared interests - that can be easy to miss or forget. We wanted to elevate human connection through technology, moving toward a model that supports social interaction quietly in the background rather than interrupting it.

Altitude was born from a single question: What if something could take on the burden of remembering, so we could stay fully present and engaged in the moment?

What it does

Altitude is a context-aware, multimodal system that observes a user's daily interactions to help them better navigate their social world.

Using a camera and a microphone, Altitude can:

  • Recognize familiar faces and identify new people
  • Build and maintain a personal database of individuals you meet
  • Listen to conversations to extract meaningful details
  • Associate new data with existing contacts

These capabilities create an evolving social memory that helps users network more effectively and "remember" what matters most, exactly when they need it.

How we built it

  • Computer Vision: Face detection and recognition to identify known and new individuals
  • Audio Processing: ElevenLabs for voice recognition to capture and transcribe conversations
  • LLMs & Reasoning: OpenAI models with LangChain to extract structured information and contextual insights
  • Database: MongoDB to store user profiles, data embeddings, and evolving social context
  • Hardware: A Raspberry Pi connected to a camera and microphone to simulate a wearable smart device
  • Backend: Python with FastAPI for real-time processing and API development
  • Frontend: React with CSS for a clean, intuitive interface to review and manage social memory

Challenges we ran into

  • Coordinating real-time data flow between vision, audio, and language models
  • Balancing responsiveness with limited hardware resources on the Raspberry Pi
  • Consistent facial recognition in diverse settings
  • Making complex AI systems feel assistive rather than overwhelming

Accomplishments that we're proud of

  • Integrating hardware, computer vision, speech, and LLMs into a single system
  • Bringing up a rapid RaspberryPi prototype to simulate real use cases

What we learned

  • Managing integrations between several technologies within one project
  • Working with facial recognition
  • Chaining tooling efficiently to achieve a multimodal system

What's next for Altitude

There are many possibilities to make Altitude more powerful:

  • Improve face recognition accuracy and concurrent tracking of people
  • Expand context awareness such as location, time, and recurring interactions
  • Explore accessibility and assistive use cases
  • Move toward a wearable form for real-world adoption

Built With

Share this project:

Updates