general magic

Inspiration

I saw a tweet, and we wanted to build something with edge compute trained on a DJX Spark.

What it does

We essentially distilled our own world model, trended on 10 minutes of video game data, and orchestrated the couple agents to play as the bad guys, like the players. We wanted a MLG commentator around the game, as well as the music also being AI-generated. Like you just play something and everything is ai generated at test-time.

How we built it

We used DJX Spark, it's a flow matching model. based on https://github.com/leofan90/Awesome-World-Models.git using the oai api sending frames to annotate them to pass them to the orchestrator/ennemy agent, some inference running on modal (mostly on the browser though) cool multi turn agent for combat, cool medieval midi music and sfx ; )

Challenges we ran into

after getting some feedback at the Nvidia booth we realized we needed to use nemotron to orchestrate the game, (game flow attacks, defence, magic -- also bad guy stuff)

Accomplishments that we're proud of

runs a fully viable world model for a small game within the browser on webgpu

What we learned

first time ever making a world model, working with webgpu, onnx,

What's next for general magic

maybe a bigger game like clash Royale or more webui that can change on the fly so that AI b2b SaaS wrappers can have ultimate conversion

Built With

• codex