KinoPilot - AR Filmmaking + AI Script Supervisor

An AR assistant director for filmmakers, powered by Spectacles + AI

Comment

Inspiration

"Kino" refers to a film-making movement that advocates the production of short-films on little to no budget, using small crews, and non-competitive collaboration. I am an award-winning film director because of this very passion found within independent film groups, and I wanted to make a tool that I would have thought to be genuinely useful to others and myself while directing. One of the major issues we face is the lack of crew members, meaning important tasks like script supervision have to be sacrificed. As a result, I wanted to make a tool that could streamline the necessary tools, move away from destroyable paper or bulky device-based solutions, and allow directors to stay engaged in their work.

Additionally, one of the suggestions given by the Spectacles track was to explore how one can create a world of their dreams. To me, there's nothing that exemplifies this more than making a movie.

What it does

KinoPilot has 3 major components:

  1. Screenplay Viewer - Filmmakers can upload the scripts of their production and flip through them digitally, allowing for them to stay immersed in their actors performances as they read along rather than looking down at a sheet of paper.
  2. Shot Notes - Review every take in real-time. Annotate the scene, shot, and take numbers with a usability rating (good, okay, bad) and add tags to denote the most common mistakes (off performances, camera troubles, or audio issues).
  3. AI Script Supervisor - Powered by Gemini, directors can use their script and the current scene/shot as context for generating prop lists, scene summaries, and character directions.

How we built it

This was made using Snapchat's Lens Studios for the Spectacles AR glasses. The AI tools are powered by Google's Gemini LLM flash model.

Challenges we ran into

My first attempts at making a dynamic shot list system via UI component instantiation failed, so I went for an more time-consuming but optimal object pooling approach

Accomplishments that we're proud of

I did this project completely solo, starting at 3pm Saturday more than half a day after the hackathon started. In the end I ended up making something that I would find genuinely useful

What we learned

There's still a huge divide in the usage of emerging technologies including AI and AR in art, so this helped me explore both practical and ethical uses of these tools for filmmaking. I've also never had the chance to build for AR glasses, so it was a fantastic opportunity to work hands-on with Spectacles and Lens Studio.

What's next for AR Script Supervisor

There are so many opportunities to expand the toolset of KinoPilot. 3D position solving you could block out scenes ahead of time. Object detection could help with continuity checks. It would also be a great tool for preproduction, allowing you to scout for locations in an non-disruptive and incognito manner.

Built With

Share this project:

Updates