Iris | Devpost

Inspiration

Most smart glasses want you to buy new frames, and then they just wait for you to ask them something. We wanted to flip that around. You keep the glasses you already wear, and they start paying attention for you.

It really clicked when we thought about leaving things behind. Everyone has walked out the door without their phone, or spent ten minutes retracing their steps looking for their keys. We wanted glasses that notice before you forget, and that remember for you when you do.

What it does

Iris slides onto the arms of the glasses you already wear. A small camera sees what you see, and a brain running on a laptop decides whether anything is worth interrupting you for. Most of the time it stays quiet, which turned out to be the most important thing it does. When something does matter, it puts a short line in the corner of your eye, and says it out loud in its own voice. You can also just ask it things, like how much protein is in the snack you're holding.

Everything Iris sees becomes a memory. Ask where you left your phone and it finds the last moment your phone was in view. The memory garden shows all of it as a glowing tree of life: every moment Iris lived is a leaf, faint where it stayed quiet and bright where it spoke, and saved memories bloom as golden blossoms. New leaves sprout while you watch. Ask about your phone and the garden flies to that blossom, which opens into a 3D scene rebuilt from that single photo.

Iris can also act on what it sees through three Fetch.ai agents that live on Agentverse and can be reached from ASI:One. One of them looks things up based on what you're looking at, like the weather or the next SpaceX launch. Another answers "where did I see that?" from your memories. The third is Iris's restraint, the part that decides whether a moment deserves an interruption at all.

How we built it

The camera is an ESP32-S3 board standing at the hinge of the right arm, and it hands over an 800x600 photo in about a fifth of a second. The display sits on the left: a tiny OLED on a XIAO ESP32C3, shining down through a magnifying lens onto a half-mirror we cut from acrylic and window film, so only the lit pixels show and they seem to float over the room. Both slide onto the glasses on zip-tie loops and run from one power bank in your pocket. Instead of a plain cursor, the display has a little animated eye that opens when you say "Iris", looks like it's thinking while it works, and blinks when it answers.

The brain is a FastAPI server in Python. GPT-6 Luna looks at each frame and suggests what to say, Claude Haiku 4.5 answers your questions, and Gemini stands by as a backup from a third provider. We picked them with a bake-off on photos from our own camera. A gate then decides how loudly to say it, if at all, and the dashboard shows that whole trace: what Iris saw, what it considered, which rule held it back, and how long each step took. On the real glasses, the first spoken word of an answer arrives in about 1.7 seconds. We designed Iris's voice ourselves with ElevenLabs Voice Design, and ElevenLabs streams every answer back as speech.

Memory runs as a Neon Function right next to a Postgres database. Each moment is stored with an embedding of its description in pgvector, and the photo goes into private Neon storage in the same step, so a memory can never lose its picture. Frames that look just like the last one are skipped before they're saved. Search looks for moments that match by meaning or name the object outright, then returns the most recent one, because "where is my phone" really means where did I last see it.

The web app is built in React and styled with Tailwind, and every screen was designed in Figma first. The phone page is how you talk to Iris, and the dashboard shows what the brain is thinking, including everything it decided not to say. The garden is a real-time 3D tree in react-three-fiber, drawing thousands of leaves at once, modelled on concept art we made first. It runs Depth Anything in the browser to turn a flat photo into a shallow 3D scene you can lean into, and it works in WebXR on a phone in a Cardboard viewer.

Challenges we ran into

The optics took most of the night. A strong magnifying lens makes text big but warps it at the edges, so we searched for the gap where the text floats sharp, and kept it in the middle of the screen. The mirror film wouldn't stick until we found the nearly invisible backing still on it, and our first acrylic pieces cracked until we learned to score and snap them instead of cutting.

Our camera's picture was upside down, and that alone made the model miss the planted mistake four times out of five. Turned the right way up, it caught it every time. Now the brain resends the camera's settings at the start of every session, so a restart can't undo it.

Then the display box ended up right in front of the camera. So we split them: the camera on the right arm, the display on the left. Nothing blocks the view, and the weight balances. Speed shaped almost everything. The brain gives memory two seconds to answer, and our first search made two model calls back to back. We changed it so the object comes straight from the words in the question, and now most searches need only one embedding call.

The network fought us more than once. Our Macs took about five seconds to find the boards by name, so we switched to plain IP addresses. Later the venue Wi-Fi blocked the port Postgres uses, which meant we couldn't reach our own database from our laptops. Because memory already talked to Neon over regular HTTPS, it kept working anyway.

Phones won't let a web page use the microphone without HTTPS, so everything runs behind a Cloudflare tunnel. The catch is that a quick tunnel gets a new address every time it starts, so our start script writes the new address into the config and rebuilds the web app on its own.

Late at night memory suddenly went down, and it took us a while to figure out why. Our OpenAI account had simply run out of credits. After that we got into the habit of testing each piece by itself instead of guessing.

Building 3D from a single photo was tricky too. One image can't show what's behind things, so pushing it too far smears everything. Keeping the scene shallow and fading out the stretched edges made it feel like a diorama instead of a mess.

And with four of us sharing one repo for 24 hours, we had to write down exactly how the pieces talk to each other and then stick to it.

What we learned

We came in thinking the hard part would be getting Iris to say smart things. It turned out the hard part was getting it to stay quiet. An assistant that talks too much is one you tune out, so most of our time went into the gate that decides when a moment is worth interrupting you, and that mattered far more than which model we used.

We also learned how much a time limit shapes a project. Once we knew the brain only had two seconds to answer, it changed how we designed memory from the ground up, not just the parts that were slow.

Turning a single photo into 3D taught us to know when to stop. Depth Anything gives you a surprisingly convincing scene as long as you don't push it too far, and keeping it shallow looked better than trying to make it perfect.

And at 3am, when memory suddenly stopped working, we learned the most practical lesson of all. Checking one piece at a time found the problem in minutes, after a lot of guessing had gotten us nowhere.

What's next

We want Iris to notice when you're heading out the door and remind you about the phone you left on the table. We've started building that, but it still needs tuning on the real glasses. We'd also like everyone to have their own private memory through sign-in, and we want to shrink the clip so it's lighter and lasts longer on a charge.