Reptrainer

Reptrainer simulates real buyer objections using Gemini Live, grounds evaluations in your actual product knowledge, and helps teams close with confidence.

Comment

Summary

Reptrainer is a real-time AI "flight simulator" for enterprise sales teams. It allows sales representatives to practice high-pressure buyer conversations using low-latency voice roleplay. The core experience is driven by the Gemini Live API, providing a seamless, natural conversation flow where the AI buyer can be interrupted, challenged, and guided just like a real prospect.

Key Features

  • ** Voice-First Roleplay**: High-fidelity, real-time voice conversations with AI buyer personas via Gemini Live API.
  • ** Whisper Coach**: A real-time HUD that analyzes your conversation and "whispers" tactical nudges and objection-handling strategies.
  • ** AI Persona Generation**: Instantly generate diverse buyer personas (from Skeptical CFOs to Decision Makers) tailored to your specific product.
  • ** Coaching Insights**: Advanced analytics that identify skill gaps (Discovery, Closing, Listening, etc) and trends across your team.
  • ** Multimodal AI Debrief**: Post-session debriefs featuring synchronized audio narration, visual slides, and interactive objection heatmaps. Generates a synchronized 4-slide presentation for a session (if the session lasted for at least 3 minutes) using Gemini's native multimodal output. Coaching text and infographic images are generated together in a single API call, producing contextually coherent visuals that directly reflect the coaching analysis.
  • RAG-Grounded Personas: Buyer personas are grounded in the user's uploaded product documentation and competitor "battle cards."
  • ** Team Management**: Role-based access control for Admins (Team Leaders) and Members, with aggregated team performance dashboards.

Technologies Used

  • Gemini Live API & ADK (via Python): For the real-time bidirectional voice agent.
  • Gemini 2.5 Flash (Multimodal Output): For unified debrief generation — produces coaching analysis text and infographic images in a single interleaved response using responseModalities: ['TEXT', 'IMAGE'].
  • Gemini 2.5 Flash: For conversation analysis, grading, and persona generation.
  • Vertex AI (Search Grounding): For deep competitor research and grounding.
  • Vertex AI Rag: Use rag corpus as for creating team knowledge-base.
  • Firebase (Auth, Firestore, Storage): For real-time state synchronization and asset management.
  • Next.js & Express: Full-stack framework for the web dashboard and proxy layer.

Findings and Learnings

  • Unified Multimodal Output + Cloud TTS: Our biggest performance win came from shifting to a hybrid debrief pipeline. We now use Gemini's native multimodal output to generate coaching text and contextually coherent infographics together in a single interleaved response, while retaining Cloud TTS for high-fidelity audio narration. This approach minimizes "lost in translation" errors between text and visuals while maintaining a polished voiceover experience.
  • The Battle for Brevity in Live Audio: Unlike text chat where detailed responses are helpful, live voice interactions demand extreme brevity. We discovered that any AI response over 15-20 seconds feels like a monologue and discourages user interaction. We had to implement strict guards in the system prompt and explicitly instruct the model to use natural pauses and keep-it-short prompts to maintain a true back-and-forth flow.
  • Tool Calling Latency vs. Immersion: Executing tools (like Google Search or competitor research) during a live roleplay creates a "dead air" gap that can break the user's immersion. We found that the most effective solution was prompting the AI to use natural "filler" phrases (e.g., "One second, let me verify that detail...") while the tool executes, and ensuring that tool results are synthesized into a single natural conversational sentence rather than recited as raw data.
  • GCP Integration: Using a unified Google Cloud ecosystem (Vertex AI + Firebase) allowed us to build complex multimodal features with minimal glue code, as the SDKs handle authentication and data flow seamlessly.

Inspiration

Sales is a high-stakes performance, but practicing it often feels either boring or "cringe." Most reps walk into critical discovery calls with sweaty palms because they haven't "repped" the scenario in a realistic environment. We were inspired by flight simulators for pilots - why couldn't sales reps have a safe, high-fidelity place to practice fumbling on pricing or handling a tough CFO? The release of the Gemini Live API was the missing link, finally allowing us to build a voice-first "simulator" that feels human, responsive, and truly reactive.

What it does

Reptrainer is a real-time AI training ground for enterprise sales.

  • Voice-First Roleplay: Practice discovery and closing calls with AI buyer personas that sound human and can be interrupted.
  • Whisper Coach: A live HUD that "whispers" tactical advice (e.g., "Pivot to ROI") while you speak.
  • Grounded Personas: AI buyers aren't just generic bots; they are grounded in your actual product docs and competitor "battle cards" using RAG.
  • Multimodal Debrief: After every session, the AI generates a personalized 4-slide "Executive Coaching" presentation with audio narration and visual infographics.
  • Performance Analytics: Team leaders can track skill gaps across the entire organization (Discovery, Closing, Objection Handling).

How we built it

We built Reptrainer as a distributed multimodal architecture:

  • Frontend: A Next.js application utilizing the Web Audio API with custom Worklets for 16kHz PCM capture and Voice Activity Detection (VAD).
  • Real-Time Brain: A Python service powered by the Google ADK (Agent Development Kit) and FastAPI, which manages bidirectional streaming with the Gemini Live API.
  • Orchestration: An Express.js API that coordinates the complex "Debrief" flow, calling Gemini 2.5 Flash for analysis and visual generation, alongside Cloud TTS for narration.
  • Infrastructure: Firebase (Auth, Firestore, Storage) provides the real-time state sync and asset hosting, while Google Cloud Build and Cloud Run ensure a seamless CI/CD pipeline.

Challenges we ran into

  • The "Echo" Effect: One of our most persistent bugs was a race condition where the AI would occasionally "double-message"—repeating its last sentence or talking over itself after a tool call. We had to implement strict state tracking in the WebSocket bridge to ensure only the most recent generation reached the user's ears.
  • Navigating GCP: As first-time users of the full Google Cloud suite, we spent a significant amount of time just learning the console's geography. Figuring out the specific IAM permissions needed to let Cloud Build talk to Vertex AI while Firebase handled the frontend was a steep but rewarding learning curve.

Accomplishments that we're proud of

  • Natural Conversation Flow: We successfully moved beyond "robotic" turn-taking. The AI handles interruptions and tangents with a level of realism that actually helps users build muscle memory.
  • The "Wow" Factor of the Debrief: Seeing a user's face when they receive a narrated, visual presentation analyzing their performance just seconds after hanging up.
  • Deep Grounding: Creating a system where a CFO persona can raise an objection specifically about a competitor's recent price hike because it was in the uploaded "battle card."

What we learned

  • Feedback is Key: Sales reps engage significantly more with visual and auditory feedback than with simple text reports.
  • Unified Ecosystem Efficiency: Staying within the Google Cloud/Firebase ecosystem allowed us to build features more seamlessly than if we were stitching together disconnected providers.
  • Coupling Context: We learned that for roleplay to be effective, the AI must "know" the product it is buying. Generic personas lead to generic practice.
  • The "Uncanny Valley" of Voice: If the AI responds too fast, it's startling; if it waits too long, the user thinks the call dropped. Finding that sweet spot of 400-600ms response window is the secret sauce.
  • Simplicity Wins: We started with a massive 30-prompt chain, but quickly learned that a single, well-grounded system prompt with clear constraints worked better than trying to over-engineer the agent's logic.

What's next for Reptrainer

  • Gemini Vision Integration: We want the AI to "see" the rep to provide coaching on eye contact, body language, and confidence signals.
  • Multi-Buyer Committee: Roleplay sessions with 2-3 different AI stakeholders (e.g., the CFO, the Technical Lead, and the Champion) in the same call.
  • Screen-Share Reaction: Enabling the persona to react in real-time to the pitch deck or product demo the rep is sharing on their screen.
Share this project:

Updates

Submission history