Sal

Your Gemini-powered sales co-pilot: prepping, coaching, and handling your follow-ups in real time.

Comment

Inspiration

Sales calls are high-stakes environments. Every minute a seller spends looking down at a notepad or wrestling with CRM fields is a minute they aren't listening to their customer. I was inspired by the idea of a "Second Brain" for the sales rep—an assistant that doesn't just transcribe the call after the meeting, but one that can advise and act (through tools) in real time.

What it does

Sal is a real-time multimodal sales assistant designed for the entire meeting lifecycle:

  • Prep: Salespeople can sync their Google Calendar and select a meeting they'd like the agent (Sal) to focus on. Once selected, Sal gains context on the meeting and is able brief the salesperson on upcoming attendees, previous notes, and outstanding action items.
  • Live Assist: During live calls, Sal listens actively as the salesperson shares their audio (and optionally their screen). An always-on Picture In Picture display provides real-time advice from Sal, and allows the salesperson to see remaining action items.
  • Proactive Action Capture: Unlike traditional AI tools that require a post-call summary, Sal not only identifies new action items as the call is happening but also marks existing action items complete so the salesperson can be sure that they covered everything they wanted to.
  • Closing the Loop: Sal facilitates a post-call debrief to capture final thoughts.

How I built it

Sal is built entirely on top of Google Cloud Platform, and the core agent interaction is built on top of the Gemini Live API (through the Google ADK).

Frontend:

A Next.js application featuring a custom multimodal capture engine and a dedicated audio player for handling Gemini's native-audio responses. The frontend handles interruption events from the Gemini Live API.

Agent Gateway:

A Python FastAPI application that proxies Websocket connections to the Gemini Live API (through the ADK's LiveRequestQueue) and hosts various RPC services (User Profile Service, Calendar Service and Agent Session Service).

Infrastructure:

The entire stack is managed via Terraform. Services are deployed on Google Cloud Run, images are pushed to Artifact Registry, secrets are managed using Secret Manager, user state (calendar events, sessions and profiles) is persisted in Cloud Firestore, user authentication (and application security) is done at the edge through Identity Aware Proxy (IAP). All APIs are strongly typed through strict Protobuf contracts (ensuring typesafety across typescript & python). The only component which isn't checked into this repo is my Global Application Load Balancer which I share for multiple projects.

The Model:

I'm using gemini-2.5-flash-native-audio

Challenges I ran into

Noting a few points below:

  • The interruption model still feels a bit slow (it takes more time than I really want it to to interrupt the model).
  • The Gemini Live API seems a bit finicky about replaying session history (for example, it doesn't seem to handle previous function calls gracefully), which meant I needed to sanitize various inputs prior to feeding it to gemini live.
  • The current gemini live model (gemini-2.5-flash-native-audio) could be better good at tool calling... I needed to keep my tools as simple as possible for it to work reliably. It was hard to get gemini to get gemini-2.5-flash-native-audio to passively listen when I was starting a meeting (even though the proactivity config did help quite a bit on that front).
  • I'd need much better error handling to have a reliable agent!

Accomplishments that I'm proud of

I truly think this is a neat and helpful product! It really shines through when Sal is proactive with tool usage. Most AI assistants wait for a prompt like "Add this to my list." Sal is different; it's instructed to listen for commitments and update the meeting record autonomously. I also think the "interactive" vs "background" agent functionality is quite interesting. Lastly, I'm quite pleased with how this entire project runs on GCP, behind IAP. Cloud Run deployments are trivial (I'm using skaffold.dev) and managing GCP infra was Terraform is a breeze (with LLMs helping out, of course). Ah, and having IAP at the edge means I can easily share this with friends without worrying about random folks blowing up my cloud bill :D

What I learned

This was my first "Live Agent" application, and even though there's quite a few rough edges it was a lot less scary than I was expecting. I think there's a very interesting paradigm shift going from a "Prompt and Response" to an "Observe and Assist" model.

What's next for Sal

On the product side, there's obviously lots of things to do:

  • Deeper CRM Integrations
  • Going beyond the single-meeting state... to have the agent keep the same context across follow ups.
  • Letting Sal "actually be present in meetings", rather than just shadowing the salesrep?

On the tech side, there's also lots of things to do! From proper error handling to compressing context when it gets too long...

Built With

Share this project:

Updates

Submission history