AURIC

Inspiration

Phones are the wrong shape for an assistant. They can't see what you see, and you have to stop what you're doing to use them. Smart glasses fix both. I wanted an assistant that lives in the glasses, keeps up with a conversation, and remembers the world for me.

What it does

AURIC is a companion for Meta smart glasses, run from an iPhone.

Conversation. Talk to it naturally. It follows the thread of the conversation and answers in your ear. You can interrupt it mid-sentence.

Visual memory. Look at something and say "remember where I put this." Ask later and it tells you where and when it last saw it.

Finding. A Hack the North badge becomes a tracker. As you get closer, the phone taps faster, like a Geiger counter. Two iPhones can also range each other directly with Apple's Nearby Interaction.

Space. Capture a room and AURIC rebuilds it as a 3D Gaussian splat you can explore.

How I built it

The app is Expo and React Native, with a native Swift module for Meta's Wearables Device Access Toolkit and Apple Nearby Interaction. A Node relay handles the reasoning: OpenAI for vision, tool use and transcription, Gemini as an alternate, and ElevenLabs for the voice. The badge firmware is Lua on an ESP32. Room reconstruction runs on FFmpeg, COLMAP and Brush, with a Three.js viewer.

Challenges

The badge radio can only be heard by another badge, so a phone can't listen to it. I turned a second badge into a receiver that streams what it hears over USB to the laptop and on to the phone.

Bluetooth glasses make you choose between good audio out and a microphone in. Using the glasses' mic drops the whole link to phone-call quality, so the phone does the listening and the glasses do the talking.

Meta's camera SDK needs a native build, so part of the app lives outside Expo Go as a custom module.

Accomplishments

One person, one weekend: a phone app, a native glasses bridge, badge firmware, an AI relay and a working 3D reconstruction pipeline, all talking to each other.

I'm proudest of how it answers. Every memory carries where it came from and when, so AURIC says "last seen on your desk twelve minutes ago" instead of guessing.

What I learned

Every sensor answers a different question. A photo tells you where something was. Radio tells you whether you're getting warmer. Keeping those separate made the assistant far more useful than blending them into one confident answer.

I used OpenAI Codex throughout the build, from the Swift bridge to the relay's tool layer.

What's next

More time on real hardware: tuning latency in live conversation, improving recall on real captures, and using the 3D map to guide you to what you're looking for.

Built With

• bluetooth
• colmap
• elevenlabs
• esp32
• expo.io
• ffmpeg
• gaussian-splatting
• gemini
• lua
• nearby-interaction
• next.js
• node.js
• openai
• react-native
• swift
• three.js
• typescript
• web-serial