Aura Whisper

Inspiration We’ve all been there: you’re on a high-stakes call with a potential client or investor, things are going great, and suddenly they hit you with a curveball objection or a hyper-specific technical question. Your mind goes completely blank, panic sets in, and you stall.

Existing sales and meeting tools are essentially "post-mortems"—they analyze the transcript and tell you why you lost the deal after the call is already over. We wanted to build something that actually saves you in the moment. The idea was simple: give every founder and sales rep a brilliant, silent co-pilot whispering the perfect answers on their screen, right when it matters most.

What it does Aura Whisper is a live, heads-up display (HUD) for your virtual meetings. It sits quietly in the background of your Zoom, Google Meet, or Teams calls, listens to the conversation, and flashes bite-sized, actionable insights directly on your screen.

If a client brings up a competitor, Aura Whisper instantly displays your battle card highlights.

If they ask about a weird compliance detail, it grabs the exact line from your company docs using a fast semantic search.

It even tracks the emotional vibe of the call, giving you a subtle visual heads-up if the client's tone shifts from interested to skeptical.

It doesn't talk for you; it just makes sure you never run out of the right things to say.

How we built it We wanted the user experience to feel completely seamless, so we built the frontend using Next.js and Tailwind CSS to keep the interface clean, lightweight, and non-intrusive.

The real engineering magic happens on the backend. We utilized the new OpenAI Realtime API over WebRTC to handle low-latency, bidirectional audio streaming. To feed the AI contextually accurate data without slowing down the pipeline, we hooked it up to a fast vector database containing product wikis, pricing matrices, and technical documentation. The closest matching company data is instantly injected into the model's working memory so it always has the right facts ready.

Challenges we ran into Latency was our absolute biggest enemy. In a live conversation, half a second feels like an eternity. If the AI takes three seconds to come up with a counter-argument, the conversation has already moved on.

We set a strict performance budget: the total time from the client speaking to the tip appearing on screen had to be under 400 milliseconds. Balancing audio chunking, network transit via WebRTC, and AI inference to hit that target took a massive amount of trial and error.

Another major headache was prompt engineering for high-pressure scenarios. Initially, the LLM wanted to generate long, beautifully structured paragraphs. But nobody has time to read an essay while pitching. We spent hours forcing the model to output 3-to-5-word punchy bullet points that you can read out of the corner of your eye while maintaining eye contact with the camera.

Accomplishments that we're proud of We managed to get the system down to true real-time performance. Seeing the UI dynamically flash the exact piece of data you need—literally a split second after the user asks for it—feels like magic. We also managed to build a custom emotion-tracking algorithm that doesn't just read the transcript text, but actually listens to the vocal energy and pacing to gauge sentiment accurately.

What we learned Working with live audio streaming taught us that building for voice is fundamentally different from building text-based chat apps. You can't just wait for a sentence to end; you have to design systems that handle streaming contexts mid-sentence.

We also realized that in UI design for live tools, less is infinitely more. A single perfect word flashing on your screen at the exact right moment is worth more than ten paragraphs of AI-generated text.

What's next for Aura Whisper Right now, Aura Whisper runs beautifully in the browser. Next, we want to build it as a native desktop application so it can capture system audio across any meeting software automatically. We are also planning to add a "live team sync" feature, allowing sales managers to update pricing sheets or objection scripts on the fly, instantly updating the AI's knowledge base for all active reps on live calls.

Built With

• artificial-intelligence
• audio-streaming
• gpt-4o
• llm
• next.js
• node.js
• openai
• openai-realtime-api
• pinecone
• python
• react
• real-time
• semantic-search
• speech-to-text
• streaming-data
• tailwind-css
• typescript
• vector-database
• vercel
• web-audio-api
• webrtc