Gen AI Mario

Sprite if Mario was a Barbie character

Inspiration

Inspired by the Mario Maker level generator, we wanted to generate a mod for Mario on runtime using Generative AI>

What it does

Generates a player-generated mod for Mario using Generative AI text to image and text to audio models

How we built it

We found a Web open-source implementation of Mario and generated a chat interface with Gradio that would allow players to create mods using Mario

Challenges we ran into

Generating Sprites using an Image-to-Image AI generator wasn't consistent and the Stability API for Image to Image was down throughout. Audio generation is not fine-tuned for Gaming

Accomplishments that we're proud of

Calling functions with OpenAI and generating prompts for other multi-modal models to consume using Open AI's GPT4

What we learned

Generating Sprites for Retro Game Mods is a solvable problem but requires experimentation & fine-tuning. Generating Audio for Games requires fine-tuning.