Mimi is an expressive, human like AI assistant you can actually talk to and accomplish things with.
She can have a casual conversation, tell you a story, remember things you've told her, manage your schedule, read a PDF or spreadsheet, explain why a stock moved, find the latest news, or remind you about something later.
Mimi is privacy first, storing your personal data always on your device.
Inspiration
Most voice assistants feel a bit like vending machines. You speak into them, get an answer, and that's it. There's no real sense that anyone is there, listening or remembering what you said before.
We wanted Mimi to feel more present. Something that reacts while you're talking, remembers the conversation you had last week, and feels closer to talking to a person than typing a question into a search box.
She started as two glowing eyes and a mouth on a dark screen. No human avatar, no attempt to pretend she's a person — just enough of a face to make the interaction feel a little more alive.
Disclaimer: We are hugely inspired by Emobot!
What it does
You click “Talk to Mimi” and just start talking. Her face has 42 different expressions, and her mouth moves with the actual audio rather than playing a canned animation.
She remembers things you tell her, and that memory stays in your own browser. You can drop a PDF onto her face and ask questions about it, ask her to search the web, or give her harder jobs like coding, financial analysis, research, or turning messy information into structured notes.
Mimi goes into an idle mode if you don't speak with her for 8 seconds and you can wake her up by just saying her name and she's ready to speak with you again.
She can also do things for you. For example, she can draft an email directly into Gmail for you to review and send, or set a reminder that arrives by email even if you've closed the browser.
Mimi remembers things that are important to you. You see them in the "brain" section of her UI. She constantly compresses her memories to delete duplicates and keep the core manageable. Users can export their memories from one browser to another one by just clicking export or import.
How we built it
We deliberately kept Mimi's interface simple. The front end is just HTML, CSS, and JavaScript — no complicated framework or build system. Her face is built to feel more alive than a normal animation, with expressions that move and settle naturally rather than simply switching from one image to another.
The conversation itself happens in real time, while a small service handles the connection and keeps track of usage. Accounts, history, and other necessary information are handled separately. Most importantly, user data lives on the user's device, with the ability to export and import it so you can move to another device without giving up control of it.
Mimi also deploys temporary agents to accomplish tasks like analyse PDF's, do research and more. Think of these as skills and we'll continue to build more as we go along.
Challenges we ran into
She kept interrupting herself. Mimi's own voice was getting picked up by the microphone and convincing her that someone was speaking. Normal echo cancellation wasn't enough, so we had to teach her to ignore the microphone while she's talking unless there's a strong enough signal that someone is actually speaking.
She told us she'd sent an email she never sent. The model prepared the email but never actually did, then confidently told the user it was done. We found versions of the same problem with research and reminders.
We also discovered that not all usage costs the same. Audio is dramatically more expensive than text, so a simple “token allowance” didn't really represent what people were using. And sometimes features that should have worked together quietly interfered with each other.
Timeouts and incorrectly formatted responses broke the flow. We taught Mimi to fail just a little more gracefully now.
What we're proud of
The biggest thing we're proud of is that Mimi is honest about what she actually did. If an email wasn't sent, she doesn't say it was. If a reminder wasn't created, she tells you. Every external action is checked against what really happened rather than what the model thinks happened.
We're also proud of how much of the system we've tested for real. Usage is measured from the actual costs on the server, Mimi finishes her sentence rather than cutting off halfway when credits run out, and reminders continue to work even when the browser is closed.
We have nearly 200 unit tests as well as a 41-step end-to-end test that runs against the real deployed system, including the database, model, and email delivery.
What we learned
Measure it, don't remember it. A lot of our hardest bugs came from things that seemed so obviously true that we never thought to check them.
We now think about the failure case first: what happens when something goes wrong, and how do we make sure Mimi recovers in an honest way?
Sometimes constraints lead to better products. We couldn't add calendar integration without going through a much more expensive security process. We ended up using email reminders using resend instead, and realised they might actually be better for some things.
Product improvement is a continual, never ending process.
What's next
We want Mimi to do more with files. The ideal experience is simple: give her a file, tell her what you need, and get the finished file back — whether that's a summary, analysis, cleanup, or some other kind of manipulation.
We're also working on making her memory easier to move between devices. Today you can already export and import your data, but we're looking for more seamless ways to do that without compromising the principle that your private information belongs to you.
We know memories get larger over time. We are working to classify them in buckets that can be retrieved on demand so months or years of conversations with her feel just as responsive.
And then there's seeing. Mimi can already understand still images. The natural next step is letting her see live video — which feels especially appropriate for an assistant whose whole purpose is to feel a little more present.
Built With
- cloudrun
- firestore
- gemini
- genai
- identity
- javascript
