EmotionTalk

💡 Inspiration

We’ve all been in those high-stakes conversations—sales calls, customer check-ins, investor pitches—where emotion shifts fast and the moment slips away. The customer’s tone changes, but we miss it. EmotionTalk was built to fix that: a real-time coach that listens, reads the room, and nudges you toward the right response when it matters most.

⸻

⚙️ What It Does

EmotionTalk passively listens to a live customer ↔ owner conversation and, in real time: • Detects emotion in the customer’s voice and displays an emoji + tone badge. • Delivers three bite-size coaching tips to help the owner react better on the spot. • Generates a post-call summary outlining what went well, where trust dipped, and next steps.

It focuses only on the customer’s voice—so feedback stays objective and never penalizes the speaker.

⸻

🧠 How We Built It • Realtime voice: [Vapi Web SDK] for mic capture + streaming; Deepgram for speech-to-text with diarization metadata. • Intelligence: [Claude AI] for tone detection, evidence extraction, and concise coaching bullets; a second Claude prompt for post-call summaries. • Backend: Node.js + Express (TypeScript) • POST /api/analyze → emotion, confidence, coaching bullets • POST /api/summary → call recap • GET /api/config → public keys for frontend (no secrets exposed) • Frontend: Lightweight HTML/CSS/JS interface with pulsing mic, emoji panel, advice list, and summary view. • Safety & polish: .env keys hidden server-side; .env.example for open-source clarity; GitHub history scrubbed clean to satisfy push protection.

⸻

🚧 Challenges • Diarization accuracy: SDKs labeled speakers differently—we had to normalize roles and filter only the “customer” channel. • Latency jitter: Partial transcripts caused flashing advice updates; we fixed it with final-transcript gating and debounce logic. • Mic configuration: Balancing “listen-only” mode with optional voice output required deep Vapi tuning. • CORS issues: Local dev between :5173 and :4000 needed secure CORS + /api/config routing. • GitHub push protection: Example keys triggered secret scans—we rewrote history safely with an orphan branch.

⸻

🏆 Accomplishments We’re Proud Of • A working live demo where emotion + coaching update instantly. • A clean, intuitive UI that anyone can read at a glance. • A modular backend ready to adapt across industries (retail, SaaS, healthcare). • Strong privacy and security hygiene—no client-side secrets, clean repo history.

⸻

📚 What We Learned • How to chain speech → text → emotion → advice in a live pipeline. • Prompt engineering for short, specific coaching with cited emotional evidence. • Why final transcript gating is key for real-time UX stability. • How to handle GitHub secret scanning and keep open-source repos secure.

⸻

🚀 What’s Next • Improved multi-speaker diarization and adaptive role mapping. • Industry “coaching packs” for sales, support, healthcare, and education. • An analytics dashboard for tone trends and trust insights. • CRM + Slack/Notion exports for instant sharing. • Multi-language support and edge-side VAD for lower latency. • Human-in-the-loop feedback to train better coaching behavior.

Built With

• claude
• css
• html
• javascript
• vapi