Eve

Inspiration

Game night is supposed to be about people coming together, but it is no secret that many of us end up pulled back into our devices. We kept seeing the same problems with friends: someone was missing so the game could not start, D&D sessions slowed down from constant rules checks, and long creative games lost momentum because people forgot details from earlier.

We were also inspired by our UPenn mentor, Professor Christopher Callison-Burch, and his work with Andrew Zhu on overhearing agents. Their thesis is that the traditional chat interface is not always the best mode for communicating with AI. Eve brings that paradigm to game night.

What it does

Eve is an overhearing AI game host for popular games like D&D, Secret Hitler, and eventually any tabletop or creative game. It listens to multiplayer conversations, uses speaker diarization to understand who said what, and stays silent most of the time in order to keep social flow.

When useful, Eve can answer rules questions, track game state, catch contradictions, recall long-session details, narrate moments, and use voice to bring NPCs or missing players to life. Eve uses graph-based memory to remember players, NPCs, locations, and campaign events across long sessions.

How we built it

We built Eve as a game-blind overhearing engine with thin game adapters. The core loop has ears for speech input, memory for long-running multiplayer context, a decision gate that decides whether to speak or stay silent, and a hands layer that surfaces answers, cards, or voice responses.

For speech, we used Deepgram streaming with diarization. For D&D rules, we built a retrieval system over the SRD with HyDE query expansion, reaching about 96% recall@3 in our evals. For memory, we built a multiplayer graph that stores entities, player beliefs, events, and relationships so Eve can retrieve the right context without sending an entire 10-hour session back through a model.

Deepgram

Deepgram powers Eve's ears, the input that the entire product depends on. Our overhearing engine listens to a live D&D table, and Deepgram's streaming Nova-3 speech-to-text is how it hears. Speed matters here: the table experiences Eve in real time, so we need transcripts back in milliseconds, and Deepgram's low latency is what makes that possible. We stream the microphone straight into Deepgram and get back real-time transcripts that feed the rest of the engine, plus speaker labels (diarization) so every line arrives tagged with who said it. We also use Nova-3's keyterm prompting tied to our live campaign memory: as the engine learns names like "Toblen" or "Strahd," we feed those names back to Deepgram so it recognizes them better next time, creating a loop where better memory means better transcription and vice versa. Deepgram slots cleanly behind our audio adapter, so the same engine works on both test recordings and a real live table. We also keep Deepgram Aura on hand as a fast text-to-speech voice, leaving an all-Deepgram path open.

Challenges we ran into

The hardest challenge was making Eve useful without making it annoying. Voice agents usually respond too much, so we built cooldowns, throttling, intent classification, and polite interruption logic to keep Eve silent unless the moment mattered.

Multiplayer memory was also difficult. Real game conversations are messy: people interrupt, joke, backtrack, contradict themselves, or refer to things from hours ago. We had to configure graph memory, speaker tracking, state extraction, and contradiction detection carefully so Eve could follow the game without hallucinating or overfiring.

Accomplishments that we're proud of

We are proud that Eve feels like a new interaction pattern. It can overhear a live game, stay quiet through most of it, and still jump in with the right rule, memory, contradiction, or voice at the right time.

We shipped working D&D and Secret Hitler modes, live voice input, speaker diarization, graph-based memory, rule retrieval, contradiction detection, a themed dashboard, and over 400 automated tests covering the engine, plugins, voice routing, and deployment path.

What we learned

We learned that for social AI, timing is as important as intelligence. We also learned that games are a perfect testbed for overhearing agents because they combine speech, memory, rules, state, creativity, and social flow in one environment.

What's next for Eve

Next, we want to add browser voice output, more games like Werewolf, Avalon, Codenames, and Catan, stronger NPC roleplay, and autonomous missing-player agents with adjustable difficulty. Long-term, we want Eve to become the AI layer that makes creative games faster to start, easier to run, and more fun to play together. And most of all bring game night back.

Built With

• anthropic-claude
• deepgram
• docker
• elevenlabs
• fastapi
• javascript
• openai
• python
• render
• sqlite