Rehearsal

Inspiration

Most high-stakes conversations—asking for a raise, addressing a messy roommate, setting boundaries with family, or managing a toxic colleague—fail not because of a lack of preparation, but because of vocal panic and emotional friction.

In high-pressure situations, people talk too fast, use filler words, end statements on an insecure rising pitch (uptalk), or choose accusatory phrasing that triggers instant defensiveness.

Pilots log hundreds of hours in flight simulators before flying passengers so their muscle memory takes over under stress. Yet, before the most critical conversations of our lives, we get zero realistic simulation. We wanted to build Rehearsal: an AI-powered, retro-tactile cognitive flight simulator where anyone can practice hard conversations with emotionally reactive AI personas before it counts.

What it does

Rehearsal lets users step into realistic interpersonal simulations with AI counterparts whose emotional defensiveness dynamically shifts based on both what you say and how you say it:

1. Stateful 3-Axis Affect Engine: The AI maintains dynamic psychological memory tracked across Openness, Defensiveness, and Trust (0–10 scale). Accusatory language spikes defensiveness; empathy and strategic questions open breakthroughs.
2. Real-Time Vocal Biometrics & Polygraph HUD: Using real-time autocorrelation signal processing ($F_0$ pitch tracking), the simulator measures speech cadence (WPM), hesitation latencies, and alerts you if "uptalk" (rising inflection) is weakening your authority.
3. Active Interruption & Cross-Talk Physics: If you monologue for too long or deliver rapid breathless arguments, impatient personas will actively interrupt you, forcing you to practice yielding gracefully and regaining composure.
4. Tape Rewind & Conversational Multiverse: Make a conversational blunder? Hit ⏪ REWIND to step back one turn, restore the counterpart’s previous mood, and test alternate branches—or explore the full dialogue tree in the Multiverse Graph.
5. Tactical Ghost Teleprompter: An in-ear co-pilot based on the Chris Voss / FBI Crisis Negotiation framework that suggests 3 calibrated pathways: Tactical Empathy, Accusation Audits, and Calibrated "How/What" Questions.
6. Post-Session Tactical Debrief & WAV Mastering: Receive an evidence-cited scorecard with transcript quotes, custom alternative phrases to say instead, acoustic heatmap scrubbers, and a client-side mastered .WAV audio tape export.

How we built it

• Frontend & Interaction Design: Built with Next.js 14 (App Router), TypeScript, and Tailwind CSS. Styled with an editorial retro-tactile aesthetic inspired by classic cassette decks, Teenage Engineering, and Swiss typography, featuring custom magnetic fluid cursors, 3D scroll reveals, and infinite marquee ribbons.
• Acoustic Signal Processing: Implemented client-side Web Audio API analyzers and mathematical autocorrelation algorithms to extract fundamental voice pitch ($F_0$), micro-tremor stability, disfluency rates, and speech pauses in real time without server latency.
• Cognitive Persona AI: Powered by Google’s Gemini API (@google/genai) with structured JSON output formatting, prompt-engineered to emulate complex human friction, defensiveness curves, and DISC/Enneagram negotiation archetypes.
• Audio Synthesizer & Tape Mastering: Built a custom Web Audio synthesizer for tactile mechanical switch clicks and an offline audio mixer that stitches turn recordings into downloadable high-fidelity WAV tapes.
• Data Visualizations: Integrated Recharts and HTML5 Canvas for real-time ECG polygraph waveforms, affect trajectory timelines, and acoustic scrubber heatmaps.

Challenges we ran into

• Acoustic Signal Synchronization: Capturing live micro-latencies and pitch slope inflection in the browser while maintaining conversational responsiveness required fine-tuning autocorrelation buffers to avoid UI thread lag.
• Realistic Psychological Pushback: Preventing the AI from becoming an agreeable chatbot. We spent significant time crafting systemic prompt constraints so personas realistically push back, remember earlier slip-ups, and require genuine tactical empathy to de-escalate.
• Stateful Multiverse Forking: Enabling instant single-turn rollback required building a snapshot state manager that cleanly rewinds turns, audio buffers, and mathematical affect vectors on the fly.

Accomplishments that we're proud of

• Delivering an end-to-end tactile experience where speech acoustics directly modulate AI emotional behavior in under 400ms.
• Designing a warm, distinct editorial light theme and retro cassette deck console that breaks away from generic dark AI dashboards.
• Creating an offline, privacy-first architecture where all voice recordings and session archives remain 100% in the user's browser.

What we learned

• The profound difference between delivering text and speaking text: acoustic delivery (pace, pause, pitch stability) often matters more in interpersonal persuasion than the exact words chosen.
• How to architect stateful multi-turn behavioral simulations with LLMs using structured emotional telemetry vectors.

What's next for Rehearsal

• Live Multimodal Video Avatars: Incorporating real-time facial micro-expression analysis via webcam to measure eye contact, tension, and blinking.
• Custom Organization Scenarios: Enabling team leads and sales managers to upload company-specific negotiation rubrics and customer dispute scenarios.
• Mobile Native Companion App: iOS/Android recording mode for quick 3-minute warmups right before stepping into important meetings.

Built With

• ai-simulation
• audio-processing
• canvas-api
• dsp
• frontend
• gemini-api
• google-genai-sdk
• localstorage
• lucide-icons
• machine-learning
• natural-language-processing
• next.js
• react
• recharts
• responsive-design
• retro-aesthetics
• speech-recognition
• state-management
• tailwind-css
• typescript
• ui/ux-design
• web-audio-api
• web-speech-api
• webflow-design