The Rizzistant

Inspiration

Back in high school, Masaki had a stalker that wouldn’t leave him alone. Ever since then, a fear of women and dates has prevented him from finding a partner. To save him from such a lonely life, we created the rizzistant. It helps him through dates even if he’s extremely nervous and dumb. AND, we added a safe word so that if meets any more weirdly violent girls, it can create an excuse to make escape.

What it does

The Rizzistant is your live AI dating coach that listens in on your date through Omi's live transcription and delivers real-time feedback straight to your phone. It monitors the flow of conversation, detecting pacing, engagement, and overall interaction quality to help you adjust naturally in the moment.

When it detects filler phrases like "yeah okay so…" or awkward pauses, The Rizzistant instantly sends you a conversation prompt tailored to the topic at hand, helping you recover smoothly and keep the dialogue engaging.

If things take a turn for the worse, The Rizzistant's escape feature has you covered. By subtly working a pre-set code word into a sentence (editable via the "edit code word" command), it will trigger a fake phone call, giving you a seamless exit from the date.

Afterward, The Rizzistant generates a comprehensive post-date report by scoring you across metrics like emotional awareness, conversational flow, engagement, humor, and chemistry. It also tracks your progress across multiple dates, analyzing where you've improved and where you still need work, helping you become smoother and more self-aware with every interaction.

How we built it

The Rizzistant is built on a FastAPI backend that processes live audio transcripts from the Omi wearable device.

Real-time Analysis Pipeline:

Omi captures conversation audio and streams transcription segments to our /livetranscript endpoint
Each segment is accumulated and analyzed by Claude 3.5 Haiku for real-time feedback
A smart deduplication system prevents spam by tracking previous warnings and context

Multi-Model AI Architecture:

Claude Haiku handles real-time feedback with low latency (max 1024 tokens)
Letta creates persistent agents for each user, powered by Claude 3.5 Sonnet, enabling cross-date context and progress tracking
Each Letta agent maintains conversation history and can reference previous dates when generating new summaries

Voice Command Processing:

Pattern detection identifies filler phrases like "yeah okay so..." to trigger contextual conversation tips
Voice commands like "edit code word [word]" or "start date" are parsed server-side via regex
Code word detection triggers the emergency exit sequence

Emergency Exit System:

Twilio API generates authentic phone calls using TwiML responses
When the code word is detected in conversation, The Rizzistant immediately calls your phone, providing you with an excuse to exit the interaction
Configurable phone numbers via voice commands for seamless personalization

Data Persistence:

In-memory session management for active dates
Letta's memory blocks maintain user goals, patterns, and coaching context across sessions
Omi's memory API archives date summaries with structured tags for external access

Challenges we ran into

The core challenge of The Rizzistant was building a context system that could provide intelligent, non-repetitive feedback both in real-time and across multiple dates. This required two distinct approaches to context engineering.

For real-time analysis, we needed Claude to understand not just the current conversation, but also what warnings had already been sent to avoid notification spam. Early versions would repeatedly alert about the same problematic topic. To solve this, we pass both the full conversation transcript and the history of previous notifications with timestamps to Claude on every analysis call.

For post-date analysis, the challenge was even more complex: we needed the AI to remember and reference all previous dates to track improvement. This is where Letta became essential. Letta creates persistent agents with memory blocks that store user goals, patterns, and coaching context across sessions. When generating a post-date summary, the Letta agent can use its conversation_search tool to query previous date transcripts and summaries.

The result is a two-tier context system: Claude with explicit conversation + notification history for real-time advice, and Letta with persistent memory agents for cross-date analysis.

Accomplishments that we're proud of

We're proud of building a genuinely helpful AI system that actually improves people's social awareness in real-time. By orchestrating multiple cutting-edge services, we created something that goes beyond surface-level "rizz tips" to provide personalized, longitudinal coaching. The system delivers actionable feedback fast enough to matter during live conversation, remembers your patterns across multiple dates to track improvement over time, and does it all with personality that makes the experience both effective and entertaining.

What we learned

We learned how to work with a bunch of cool AI technologies like Letta for persistent memory management and Claude for intelligent language processing, as well as backend technologies like FastAPI for building real-time APIs. Integrating Twilio for telephony and Omi for live transcription also gave us hands-on experience with multiple external services and how to orchestrate them into a cohesive system.

What's next for The Rizzistant

Better speaker detection to accurately differentiate between you and your date, ensuring feedback is based on the correct attribution of who said what. Beyond that, we want to implement multimodal real-time analysis that goes beyond just transcription to analyze tone, pacing, nervousness, speech cadence, or conversation dynamics to provide even more nuanced coaching.