Cue Beats

What if AI could turn your film idea into a complete creative brief—with plot beats, visual moods, and a soundtrack—in seconds?

Comment

Inspiration

The inspiration for Cue & Beats came from the fragmented and time-consuming nature of creative research in the film industry. Creative professionals often have to juggle multiple platforms like IMDb, Unsplash, and public domain archives to gather the necessary tonal and cultural references for a project. We saw an opportunity to create a streamlined, data-driven tool that could consolidate this process, and we were inspired by the potential of pairing Qloo's Taste AI™ with Large Language Models to surface culturally relevant, actionable creative suggestions.

What it does

Cue & Beats is a web application that acts as an "all-in-one creative accelerator." Users can input a "story seed"—ranging from a logline to a scene description—and instantly receive a comprehensive creative brief. This brief includes:

  1. Narrative Sparks: LLM-generated plot beats, subplots, and character arcs informed by Qloo's cultural affinity data.

  2. Moodboard Briefs: Curated visual and audio reference suggestions, including open-access images, color palettes, and royalty-free music snippets.

  3. Soundtrack Mixer: A tailored playlist of 8-10 tracks, ordered to match the narrative beats, with placement notes for diegetic vs. non-diegetic use.

The tool is designed to augment human creativity by providing curated references rather than generating full scripts.

How we built it

We built Cue & Beats as a zero-cost hackathon project using entirely free-tier APIs, open-source libraries, and freely available services.

Frontend: We used React with TypeScript, styled with Material UI.

Backend: We integrated with the free-tier Qloo Taste AI™ API and used Gemini LLM API for narrative and soundtrack generation.

PDF Generation: We implemented a client-side PDF export feature using the jsPDF library.

The development process followed an agile one-week sprint methodology, with a focus on iterative prompt engineering, automated testing, and a clear set of milestones to track progress.

Challenges we ran into

  1. Time Constraints: The project had a tight one-week timeframe, which required an efficient agile methodology and a focus on building a demonstrable proof-of-concept.

  2. Audience Alignment: A key challenge was ensuring that the LLM-generated content was truly informed by data and aligned with cultural trends, which was solved by integrating Qloo's Taste AI™ data.

  3. Prompt Engineering: Iteratively tuning the LLM prompts to produce high-quality, relevant narrative and soundtrack suggestions was a key technical challenge.

Accomplishments that we're proud of

  1. Proof-of-Concept: We successfully built a powerful proof-of-concept for an integrated creative accelerator within a two-week timeframe.

  2. Zero-Cost Implementation: The project was built entirely using free-tier and open-source tools, demonstrating a highly efficient and resource-conscious approach.

  3. Enhanced Pitch Quality: The application successfully delivered professional creative briefs with story and soundtrack suggestions in a single package.

  4. Ethical AI: The tool augments, rather than replaces, human creativity by providing curated references, which is a significant ethical accomplishment.

What we learned

  1. The power of combining data-driven insights (from Qloo) with the generative capabilities of LLMs to create a truly valuable tool.

  2. The importance of prompt engineering and iterative testing to achieve the desired output from an LLM.

  3. Best practices for building a full-stack, mobile-first web application under a tight deadline, leveraging modern frameworks and free-tier services.

  4. The value of a clear development methodology, from wireframing to automated deployments.

What's next for Cue Beats

  1. AI-Generated Storyboards: Integrating open-source image models like Stable Diffusion to generate scene thumbnails.

  2. Color-Grading Suggestions: Using computer vision analysis (via OpenCV) of reference images to suggest color grading palettes (LUTs).

  3. Collaborative Mode: Adding real-time editing features with WebRTC and open-source CRDT libraries to allow for multi-user collaboration.

  4. Custom Affinity Profiles: Allowing studios to upload their own proprietary datasets to create custom taste profiles, enabling a deeper level of personalization and partnership.

Built With

Share this project:

Updates