Tell

Inspiration

A story can sound convincing until someone asks the right question. We wanted to turn that social moment into a hands-free, inspectable AI interface: speak a claim, hear a specific follow-up, and see exactly what evidence changed the assessment.

What Tell does

Tell is a single-round voice and camera bluff game. A player tells a true or invented story aloud. Gemini Live listens, receives webcam frames, asks adaptive questions, and publishes structured assessments during the conversation. A live chart shows its changing suspicion score, with evidence cards that quote the player’s actual words. The player can end the round for a separate final Gemini review that returns likely bluff, likely truthful, or insufficient evidence.

How we built it

We used Next.js and TypeScript for the app; the Google GenAI SDK and ephemeral credentials for the browser’s Gemini Live session; Web Audio and AudioWorklet for microphone capture and playback; and Recharts for the evidence trail. MediaPipe measures approximate blink and head-motion cues locally. Gemini can use visible camera context or those cues to shape a follow-up, but they never determine the suspicion score. The heartbeat is a simulated effect driven only by cited story concerns, not a measured pulse. A separate Gemini analysis reviews the completed transcript without previous scores, while runtime validation rejects evidence quotes absent from player speech.

Challenges and what we learned

Live audio, transcription, camera frames, and assessment tool calls can arrive in different orders. We made the round controller reconcile those events, ignore late responses from old rounds, and keep quoted evidence traceable. We learned that an engaging AI interface needs honest uncertainty: consistency alone cannot prove a story true, and facial or speech behavior cannot prove deception. Tell is a consensual game, not a scientific lie detector.

Built With

• next.js