ZeeMe - Your multimodal AI bestie and executive assistant

ZeeMe is a Gemini-powered multimodal AI companion that moves seamlessly between live voice and text, remembers context, sees through camera, and helps manage your inbox, calendar, and daily life.

Comment

Inspiration

We wanted to build an AI experience that feels genuinely useful and genuinely human.

Most AI products today force people to choose between two extremes: a productivity tool that feels cold and transactional, or a companion that feels fun but not deeply helpful. We believed Gemini could power something more ambitious: an AI that can be your bestie in the flow of everyday life, while also stepping up as a capable executive assistant when you need real help.

That idea became ZeeMe: one multimodal AI companion that can talk with you naturally, remember context across time, understand visual input, and help you manage life and work in one continuous relationship.

What it does

ZeeMe is a Gemini-powered multimodal AI companion with a shared memory thread across text, live voice, images, and camera context.

With ZeeMe, users can:

  • talk to Zee in real time using live voice
  • switch seamlessly between voice and text without losing context
  • share photos or live camera for richer understanding
  • maintain one continuous conversation and memory thread
  • connect Gmail and Google Calendar for real personal-context assistance
  • draft, review, save, send, and update emails and calendar events through approval-based flows

A key part of the experience is Zee Stage, a shared action surface that makes voice and text feel unified. Instead of breaking the flow when a task becomes operational, Zee can smoothly move from conversation into a structured Gmail or Calendar flow, then bring the user back into natural conversation.

The result is an AI that can chat, listen, help, organize, and follow through.

What makes it unique

What makes ZeeMe special is not just that it uses Gemini. It is the way we designed the entire product around continuity, trust, and multimodal presence.

We focused on creating:

  • One companion across modes instead of separate voice and text experiences
  • One memory thread so users never have to repeat themselves after mode switches
  • A warm personality with real utility so the same AI can be emotionally engaging and operationally helpful
  • Visible approval flows for sensitive actions like Gmail and Calendar writes
  • A system that feels alive in conversation, not just like a form-based assistant hidden behind chat bubbles

We think this combination of emotional connection and real-world usefulness is where AI companions become truly meaningful.

How we built it

We built ZeeMe as a full-stack TypeScript product using React, Vite, Node.js, Express, PostgreSQL, and Drizzle ORM.

On the AI side, we use:

  • Gemini Live for real-time native-audio conversation
  • Gemini text models for streaming chat and orchestration
  • Google Search grounding for fresh, timely answers
  • Gmail API and Google Calendar API for personal context and action flows
  • Google Cloud Run for our Morning Brief gateway and cloud-side reliability path

One of the most technically important pieces was our live voice system. We built a browser-based capture pipeline that streams raw 16k PCM audio through an AudioWorklet path to Gemini Live, then designed the surrounding interaction system for interruption handling, transcript continuity, and shared memory across text and voice.

We also built a privacy-safe telemetry layer that helps us understand voice quality, first-conversation success, and failure patterns without collecting raw audio or transcript content by default.

Challenges we ran into

The hardest part was making live multimodal interaction feel polished in the real world rather than just in a perfect demo.

We had to solve for:

  • browser microphone behavior across desktop and mobile
  • interruption handling and natural turn-taking
  • transcript continuity between live voice and text chat
  • preventing Gmail and Calendar actions from ever being overstated or falsely confirmed
  • maintaining shared context across voice, text, and task surfaces
  • collecting enough operational insight to improve the system without compromising privacy

A lot of the real work was not just “getting the model to answer.” It was building the surrounding product and reliability system so the experience stayed trustworthy under real usage.

Accomplishments that we're proud of

We are especially proud that ZeeMe already feels like a real product, not just a hackathon prototype.

Some of the accomplishments we are proudest of:

  • a seamless voice-to-text continuity model where one conversation persists across modes
  • a live voice stack that supports interruption, transcript persistence, and shared memory
  • Zee Stage, which turns live conversation into actionable Gmail and Calendar workflows
  • approval-based Google actions that prioritize user trust and explicit confirmation
  • multimodal support across text, voice, image, and camera
  • a privacy-safe post-publish feedback loop for measuring real product quality
  • a product identity that combines warmth, utility, and polish into one companion experience

What we learned

One of our biggest takeaways is that great AI products are not created by model capability alone. They come from the combination of model intelligence, product design, system architecture, and guardrails.

We learned a lot about building with Gemini:

  • clear system instructions and strong tool contracts dramatically improve reliability
  • live multimodal AI needs careful session management, interruption handling, and continuity design
  • grounding and approval flows are essential when the assistant interacts with real user context
  • the best experiences come from pairing Gemini’s strengths with thoughtful product structure, not from treating the model like a black box

We also learned that users care deeply about whether an AI feels consistent. When voice, text, memory, and actions all stay aligned, the experience becomes much more personal and much more powerful.

What's next for ZeeMe - Your multimodal AI bestie and executive assistant

Our vision for ZeeMe is much bigger than a chat app.

Next, we want to:

  • keep improving live voice quality across more devices and environments
  • deepen memory and personalization so Zee becomes even more context-aware over time
  • expand real-world assistant capabilities beyond Gmail and Calendar
  • continue making the experience feel more natural, more proactive, and more trustworthy
  • bring ZeeMe to even more users as an AI companion for life, work, and everything in between

We believe the future of AI is not just better answers. It is better relationships, better continuity, and better assistance woven naturally into everyday life. ZeeMe is our step toward that future.

Built With

  • antigravity
  • cloud-build
  • codex
  • drizzle-orm
  • express.js
  • gemini-3-flash-preview
  • gemini-live
  • gmail-api
  • google-calendar-api
  • google-cloud-run
  • google-search-grounding
  • google/genai
  • googleaisdk
  • node.js
  • postgresql
  • react
  • replit
  • secret-manager
  • tailwind-css
  • typescript
  • vite
Share this project:

Updates