A Study in Scarlet: An Interactive FMV Mystery

Step into Victorian London as Dr. Watson and uncover the first case of Sherlock Holmes in an AI-powered full-motion video mystery. Every decision shapes the truth, can you solve the murder?

Comment 2

Inspiration

After reading Sir Arthur Conan Doyle’s A Study in Scarlet, (first published in Beeton’s Christmas Annual in 1887 and now in the public domain) I wanted to bring A Study in Scarlet (that is in the public domain) to life as an interactive cinematic experience that blends classic detective storytelling with modern AI tools.

I reinterpreted the story from the ground up including building a FMV React game engine. Every scene, line of dialogue, character description, and performance cue was newly written to preserve the tone of the original while adapting it to an interactive format. The goal was to merge film, game design, and generative AI into something immersive, a story that feels authored by both player and machine. The Victorian setting, moral tension, and psychological realism of Conan Doyle’s world inspired me to explore how far AI-assisted storytelling could go.

What it does

A Study in Scarlet: An Interactive FMV Mystery lets players step into the role of Dr. Watson and make narrative decisions that change the investigation. Using Veo3 AI actors and AI-generated cinematic transitions, each scene adapts to the player’s choices — revealing different clues, dialogue, and emotional outcomes. It’s part mystery, part film, part experiment in narrative control.

How I built it

The project is built as a React web app that handles branching logic, scene transitions, and variable states. For the game and art creation I spent approximately 220 hours of development, spanning scripting, video generation, React app coding, and audio/visual integration.

  • Frontend: React, TailwindCSS, and JSON scene mapping
  • Video: Live-action FMV clips with AI-generated filler shots (Veo 3, Seedream, Imagen)
  • Audio: ElevenLabs voice design and Adobe Audition for adaptive sound
  • Game logic: State-based progression, player choice tracking, and inventory flags

Challenges I ran into

Synchronizing video playback with interactive logic was the hardest technical problem. Browser memory handling and fullscreen playback caused looping issues early on. Blending AI footage with live-action required detailed prompt engineering and color grading to maintain continuity. Maintaining character consistency was also challenging. I started building the game in early August when there was just 8 second video generations with first frame in Google Flow.

Another challenge was having to re-run clips several times to get the expected result. I generated hundreds of Veo3 videos!

Accomplishments that I'm proud of:

  • Created a fully playable AI-enhanced FMV mystery prototype
  • Combined traditional filmmaking and generative AI into a cohesive story
  • Built a scalable branching framework for future episodes

What I learned

I learned how to merge creative storytelling with modern AI pipelines, balancing narrative structure, pacing, and emotion across machine-generated and filmed content. This project also deepened my understanding of performance optimization and seamless UX design for FMV-based games.

What's next for A Study in Scarlet: An Interactive FMV Mystery

I’m exploring other genres that fit the FMV storytelling style, from sci-fi to psychological dramas and survival thrillers. The goal is to push AI-assisted filmmaking further and show how interactive cinema can adapt to any world or story.

Built With

  • adobe
  • adobeanimate
  • adobeaudition
  • adobephotoshop
  • adobepremiere
  • chatgpt
  • coggle
  • elevenlabs
  • gemini
  • google-cloud
  • ideogram
  • javascript
  • leonardoai
  • ltxstudio
  • lumaai
  • midjourney
  • react
  • runway
  • sora
  • suno
  • tailwindcss
  • veo3
  • vite
Share this project:

Updates