OpenExp: Sales Memory That Learns From Deal Outcomes

AI sales memory that learns from deal outcomes — 3-channel video analysis + Q-learning ranks what actually closes deals

Comment

Inspiration

AI agents don't learn. Every conversation starts from zero — you either dump your entire history into the context window (burning tokens and money) or you lose critical context between sessions. For sales, this means call recordings, CRM data, and emails exist in silos. Nobody connects them. Nobody tracks what actually predicted a won deal vs. a lost one.

## What it does

OpenExp is a vector knowledge base with reinforcement learning. All sales data flows into one Qdrant collection — video call insights (via CallMind), CRM updates, emails. Each memory gets a Q-value that updates when you record deal outcomes:

  • Deal won → reward +1.0 → all associated insights get Q-value boost
  • Deal lost → reward -1.0 → Q-values decrease
  • Formula: Q = Q + 0.25 × reward (clamped to -0.5...1.0)

Search results are ranked by hybrid score: 60% semantic similarity + 40% Q-value. Over time, insights that predict success rise to the top. Your AI agent always retrieves the most empirically valuable context first.

CallMind is one data source — it analyzes sales call videos through 3 parallel Gemini channels: transcript (pain points, objections, budget), visual (body language, facial expressions), and fusion (cross-modal contradictions like "said yes but crossed arms").

## How we built it

  • Gemini 2.5 Flash — Native multimodal video understanding via Files API. Upload once, query 3× in parallel. No frame extraction — Gemini sees motion, gestures, expressions natively.
  • Qdrant on DigitalOcean — Vector database with FastEmbed (BAAI/bge-small-en-v1.5, 384-dim). Hybrid scoring combines vector similarity with Q-values.
  • Q-Learning — Reinforcement learning loop on top of vector memory. Deal outcomes propagate reward signals through all associated memories.
  • Unkey — REST API key management. Register via web, authenticate all programmatic access.
  • Claude Code — AI agent that orchestrates everything: reads/writes to the knowledge base, generates call prep briefings, suggests deal strategies.
  • Docker Compose — Production deployment: Qdrant + FastAPI on a single DigitalOcean Droplet.

## Challenges we ran into

  • The "unkey" pip package trap — pip install unkey gives you a Python linting tool that removes .keys() calls, NOT the Unkey API SDK. Discovered this mid-hackathon. Rewrote to use plain HTTP (httpx) against the Unkey v2 REST API.
  • Gemini structured output — Getting reliable JSON from multimodal video analysis required distinct system prompts per channel with strict output schemas.
  • Q-value persistence — Docker container rebuilds wiped Q-values. Solved with volume mounts + a rebuild_q_cache() function that recreates Q-state from Qdrant metadata on startup.
  • Cross-modal fusion prompting — The most valuable insights come from contradictions between channels (verbal positive + body language negative). Designing the fusion prompt to detect these was the hardest part.

## Accomplishments that we're proud of

  • Q-learning actually works — after just 2 deal outcomes (1 won, 1 lost), the system correctly differentiates: won-deal insights at Q=+0.25, lost-deal insights at Q=-0.25. Search results visibly re-rank.
  • Cross-modal fusion — detecting "said yes but body language says no" produces the highest-value sales signals
  • Full working pipeline in one day: upload video → 3-channel Gemini analysis → vector storage → Q-learning → ranked call prep briefing → REST API
  • Open-source: both OpenExp and CallMind

## What we learned

  • Gemini's native video understanding detects body language cues humans miss — micro-expressions, posture shifts, gaze aversion
  • Q-learning with alpha=0.25 creates meaningful signal differentiation after just 2-3 deal outcomes — you don't need thousands of data points
  • The cross-modal fusion channel consistently produces the highest-value insights: verbal-visual contradictions are the strongest predictors of deal failure
  • Vector memory + reinforcement learning is a general pattern — it works for any domain where you have outcomes to learn from

## What's next for OpenExp: Sales Memory That Learns From Deal Outcomes

  • MCP server — so any AI agent (Claude Code, Cursor, etc.) can read/write to the knowledge base natively
  • Live call analysis — real-time processing during calls, not just post-call
  • Multi-source auto-ingest — CRM webhooks, email monitoring, calendar events flowing in automatically
  • Team-level Q-learning — shared Q-values across sales reps to build institutional intelligence
  • Enterprise deployment — managed Qdrant clusters on DigitalOcean for teams

Built With

Share this project:

Updates