Omnimaster

AI-based D&D Dungeon Master that creates a story based on a dynamic knowledge base that the AI builds itself. "The key to the chaotic unknown"

Comment

Inspiration

  1. AI-controlled Dungeon & Dragons game.
  2. Interactive AI-based Dungeon & Dragons gameplay.
  3. Players are going to be treated fairly by the AI game master as the AI creates the rules for the game (the rules are dynamic).

What it does

  1. Dungeons & Dragons
  2. Modifiable gameplay
  3. AI can make rules.

How we built it

We architected Omnimaster using a modern stack—React and Vite for the frontend, Supabase for persistent data and edge logic, and Medo as a unified gateway. We integrated Gemini 2.5-flash as the narrative "Game Master" and Kling AI as the visual "Artist," creating a seamless, multi-modal D&D experience that combines dynamic text narration with on-the-fly environment generation.

Challenges we ran into

Our primary hurdles involved managing unstable AI streaming connections and "chunking" bugs, where data packets would cut off mid-stream, breaking the UI. We also faced complexities in enforcing strict D&D rules—AI models are naturally prone to "people-pleasing," so we had to engineer a "Referee" prompt architecture to force the system to deny illegal moves and strictly enforce the action economy.

Accomplishments that we're proud of

We successfully transformed the AI from a standard chatbot into a disciplined, impartial D&D Referee. We also achieved a true multi-modal workflow, where the AI narrative pipeline now automatically triggers visual generation, allowing the game to show—not just tell—the party what they are experiencing.

What we learned

We learned that LLMs cannot effectively manage game state alone; they require a deterministic logic layer and structured memory management to stay consistent. We also discovered that "Referee Prompting" is a specific skill—explicitly instructing an AI to say "no" and adhere to strict rule-sets is the key to creating a challenging and fair gaming experience.

What's next for Omnimaster

We are focusing on user experience, specifically adding a non-mandatory tutorial system and an interactive "Hint Mode" to guide new players without breaking immersion. Long-term, we aim to expand our persistent world-building features and build a high-fidelity combat HUD that visually links the action economy directly to the player's UI.

Built With

Share this project:

Updates

Submission history