RoundZero: Multimodal AI Interview Coach

Inspiration

Technical interviews are incredibly stressful. Developers and job seekers often practice algorithms or behavioral questions alone, but solitary practice lacks the pressure, unpredictability, and pacing of a real conversation. While static AI mock interviews exist, they feel robotic and asynchronous.

We were inspired by the new Gemini 2.0 Multimodal Live API to build an elite, real-time AI interview coach that actually feels human—an agent that can see your screen, hear your tone, interrupt gracefully, and dynamically adjust its questions based on your performance.

What it does

RoundZero is a next-generation multimodal AI interview coach. It conducts live, bidi-streaming technical and behavioral mock interviews.

🎙️ Natural Conversation: By utilizing the Gemini Live API, the interview feels perfectly natural, with interruption awareness and low-latency voice interaction.
👁️ Visual & Multimodal Presence: It tracks your state and context via frame capture (Computer Use/Vision) so it can see your coding environment or whiteboard.
🎭 Dynamic Personas: Are you preparing for a friendly culture fit or a grueling technical deep dive? Toggle between "Buddy" (supportive) or "Strict" (high pressure) interviewers.
🧠 Smart RAG Question Engine: Questions aren't hardcoded. It uses semantic search to pull relevant SWE, Leetcode, and HR questions dynamically based on the flow of the conversation.
📊 Insightful Reports: Post-interview, it provides rich, actionable feedback on your performance.

How we built it

We architected RoundZero using a modern, scalable, real-time stack:

The Brain (AI Layer): We built our core agent logic using the Google ADK (Agent Development Kit), heavily relying on the Gemini 2.0 Flash (Multimodal Live API). This handles the core reasoning, bidi-audio streaming, and visual analysis.
The Backend: A high-performance FastAPI (Python 3.12+) backend hosted on Google Cloud Run. It manages the WebSocket connections to handle raw PCM16 16kHz audio streams and frame ingestion.
The Intelligence (RAG): We embedded thousands of real-world interview questions using OpenAI embeddings and stored them in a Serverless Pinecone vector database.
The Data & Auth layer: We utilize Neon (Postgres) with Native Auth to manage secure user sessions and store historic performance scores.
The Frontend: A sleek, responsive React 19 + TypeScript interface styled with Tailwind CSS, capturing audio buffers and canvas frames in the browser.

Challenges we ran into

Handling real-time bi-directional audio streaming over WebSockets was our biggest hurdle. Capturing raw audio from the browser, converting it to PCM16 at 16kHz, sending it via FastAPI to the Gemini Live API, and playing back the AI's response with minimal latency required deep synchronization and custom AudioWorkletNode implementations.

We also faced challenges mapping the Google ADK constructs to a web-based frontend since the framework is extremely bleeding-edge. Ensuring the agent persisted its "interview state" without hallucinating mid-conversation required meticulous prompt engineering and state management.

Accomplishments that we're proud of

We successfully achieved sub-second conversational latency, making the AI feel genuinely present. By successfully merging RAG (Pinecone) with the Gemini Multimodal Live API, we created an agent that doesn't just chat, but grounds its conversation in a strict, professional interview rubric.

What we learned

We learned the immense power of Native streaming LLMs. The paradigm shift from standard "Request/Response" HTTP generation to continuous, stateful WebSocket streaming unlocks completely new UI/UX possibilities. We also learned how to effectively manage audio buffers and optimize browser canvas frame capture for multimodal vision.

What's next for RoundZero

Post-Interview Hall of Fame: Deep integration with MCP GenMedia (Imagen 3 and Chirp) to generate custom success certificates and high-fidelity audio performance summaries summarizing user growth.
IDE Integrations: Creating a VS Code extension so the agent can literally sit inside your IDE during the technical mock interview.
Advanced Emotion Tracking: Correlating voice stress and facial micro-expressions with specific interview questions to give users unique behavioral insights.