Undercover Video Maker

Undercover clues, placed at the exact second they’re said.

Comment

Inspiration

I make short social videos of my friends playing Undercover, the word game where everyone gets a similar word except one player. The format works because viewers can see each player's secret word above their head and read the clue each player says. Placing those texts by hand in a video editor is slow: for every clue, you scrub the timeline, find the exact second it is spoken, and position the text under the right person. I wanted a tool that does the repetitive part, so I can spend my time on posting instead of editing.

What it does

Undercover Video Maker is a local Windows web app for fixed-shot clips of three players.

  1. Upload a 30–60 second MP4.
  2. Words: click above each player's head to place their word (for example asile, asile, prison).
  3. Clues: type the clues in the order they are said, each with the player who said it (psychiatrique, Left).
  4. Transcript: the app transcribes the French speech and finds the exact moment each clue is spoken. I can review the matches, nudge timings by 0.1 s, or change a player.
  5. Render: it produces an MP4 with the player words above their heads and each clue under the right player, at the right time.

Everything runs locally: no account, no API key, no database.

How we built it

  • Planning first. I used the Devpost Learn Skill Pack in a coding agent. It interviewed me and generated scope.md, prd.md and spec.md, which are in the repository (devpost/).
  • Stack: Python 3.13, FastAPI, plain HTML/CSS/JavaScript, faster-whisper (small model, CPU, French, word-level timestamps) and FFmpeg with ASS subtitles to burn the text into the video.
  • Alignment: my list of clue words is given to Whisper as hints, then matched in order against the transcript with accent-insensitive fuzzy matching. Short words need an exact match, and a word can be matched across two or three adjacent tokens when Whisper splits it.
  • Build order: I tested the riskiest parts first (transcription and subtitle burn-in on Windows), then built the app in small slices, with a commit after each verified step.

Challenges we ran into

  • Whisper on a noisy conversation. Laughter and loud voices cause errors, and rare words are misheard ("Monopoly" became "bonnopoli", "abandonné" became "Un bon donné"). The word list and fuzzy matching fixed most of this, and doubtful matches are flagged in the review screen.
  • Timing. Whisper sometimes stretches a word over the silence before it. "Monopoly" matched a 1.3 s interval and appeared about a second too early, so for long intervals the display time is now anchored near the end of the interval, with the settings in a config file.
  • My first design was wrong. I started with full sentence subtitles. After seeing the first render, I realized I wanted only the key clue word under each player, so I changed the design and updated the planning docs.
  • Automatic speaker detection failed. I tried to detect who is speaking from motion around each player. It was right only 2 times out of 4, so I dropped it and replaced it with an ordered input (word + player).

Accomplishments that we're proud of

  • A complete end-to-end workflow that works on my own real clip: upload, place words, align clues, review, render, download, and re-render after a correction.
  • All four test clues were found at the right time in my sample, including two words that Whisper misheard.
  • It is fast enough to be useful: about 92 seconds for a first run on a 58-second clip, and about 16 seconds for a corrected re-render, measured on one local run.
  • I made a decision from data instead of hope: I measured the motion-based speaker detection, saw that it failed, and dropped it.

What we learned

  • Plan before you build, and test the riskiest part first. The early Windows transcription and burn-in spike told me the idea was feasible.
  • Check results on real data. The motion detector looked plausible in its numbers, but only watching the clips showed it was wrong.
  • Cutting scope is a feature. A manual speaker input and a simple review screen let me finish a complete workflow.

What's next for Undercover Video Maker

  • Suggest the clue words automatically, with an LLM or rules, instead of typing them.
  • Revisit automatic speaker detection with better signals than motion, such as voice separation.
  • Support other layouts and more players, and speed up transcription (smaller decoding beam, GPU).

Built With

Share this project:

Updates

Submission history