Tilt

Mafia on a video call with five AI players that read your face, voice, heart rate and words to catch you lying. You just talk, and afterwards Tilt shows you your tells.

Comment

Inspiration

Mafia is a social deduction game where a few hidden mafia players lie their way through the group while the villagers try to work out who they are and vote them out. It’s all about reading people, so we asked what happens when the players can read you better than you can hide.

What it does

Tilt is a mafia-style party game played over a video call with five AI players, each with their own voice and portrait. You draw a card to become mafia or a villager, then you just talk, no typing or clicking. A four-question warm-up sets your baseline and gives the model labelled examples of your personal tells. During the game, the AI town combines four signals to decide who to accuse or vote out:

  • Face: blinks, brows, lips, head movement
  • Heart and breathing rate: read from your webcam
  • Voice: pitch, loudness, pauses
  • Words: hedges, fillers, denials

Afterwards you get a review showing which answers and votes were right or wrong, your biggest tells, and how to improve.

Does it actually work?

Yes, and we measured it:

  • The face model, tested on people it had never seen, scores AUC 0.568 (95% CI 0.525 to 0.608) across 172 people and 1,189 recordings, permutation p = 0.005.
  • The warm-up matters: with four answers the detector reaches AUC 0.98 (0.99 with six), but without a warm-up it is at chance (0.47).
  • With clear tells, the mafia gets voted out 94% of the time with the detector versus 9% without.
  • Innocent players with no tells are framed no more often with the detector than without (21% vs 21%), so it doesn't unfairly punish honest players. Charts: https://claude.ai/artifact/9XPhtLsqTP3PsanYvXShBn

How we built it

  • Frontend: JavaScript, Vite, MediaPipe Face Landmarker, Web Audio API and in-browser speech recognition
  • Backend: Python and FastAPI, with a Bayesian logistic regression model that updates on each player's own warm-up answers
  • Face model: trained on the I3D deception dataset (CC BY 4.0)
  • ElevenLabs: character voices and narration
  • Gemini API: judges answers for deception, writes the AI players' lines and the post-game review, and transcribes answers if browser speech recognition fails
  • Presage: heart and breathing rate from short webcam clips
  • Tiger Data: Postgres storage for utterances, using a hypertable schema

We're entering the HRT prize for Best Use of Data for Predictions and Decision Making: every AI decision is a probability estimate that updates with the player's own data.

Challenges we ran into

We fixed three problems in the published landmark dataset, and fixed a leak where the detector was learning the player's role mid-game instead of their tells. Fusing four noisy signals into one probability, and personalising it from only a few warm-up answers, was the core difficulty.

What's next

More players and roles, and a hosted version.

Team

Ismail (game, interface, Presage integration) and Arij (data science, AI town, voices).

AI usage disclosure

Gemini and ElevenLabs are used inside the game as described above. Claude Code (Anthropic) was used to help write the code.

Built With

Share this project:

Updates

Submission history