Inspiration
ai-and-chill is an easy-to-use platform where you and AI become co-writers in the world's most exciting film studio.
What it does
- Users can submit movie ideas, sparking AI to generate captivating movie covers and descriptions. - -- -
- Leveraging a 'swipe to like' feature, the AI learns user preferences, tailoring future content to viewer tastes.
- Top-rated movie concepts are brought to life as 60-second AI-rendered films, accelerating the validation process for movie ideas.
- With ai-and-chill, enjoy a future where movie creation is personalized, interactive, and endlessly innovative.
How we built it
- Generate the movie script with ChatGPT3.5 and built and Agent to format it in a JSON template by doing multiple calls with sufficient context. The JSON structure is divided into acts, sequences, scenes and shots. Each item contains extra information such as setting, shot_type, characters, dialogue, etc.
- Looping through the formatted JSON and craft shot-specific prompts.
- Looping through the final prompts to create clips by combining Stability.ai animation API, AudioStack text-to-speech API, and Open AI text-to-image. I downloaded some open-source audio files for the soundtracks and used OpenAI to help me identify the best-fit track for a scene.
- Compile the video by stitching together all the clips. For this, I used various Python libraries including pillow, moviepy and others.
Challenges we ran into
By far the most effort went into prompt engineering.
The first challenge was to fit the movie script into a long JSON template. Given that the template is longer than the supported context window, incremental API calls with the sufficient context of the movie, scene, and shots were required.
The second challenge is building prompts and validation to ensure the movie follows a storyline.
Accomplishments that we're proud of
Very excited that a movie can be generated in this way.
What we learned
Prompting and formatting is type consuming. It would be cool to have and API Agent that simply takes and objective and fits it into the provided JSON template.
What's next for ai-and-chill
- Launch it in the open
- Pay for movie generation
- Pay to upload more movie ideas
Log in or sign up for Devpost to join the conversation.