Inspiration

As both a passionate gamer and an AI enthusiast, I've always believed that the soul of any great game lies in the story it tells. When I first experienced Gemini's language expressiveness and powerful tool-calling capabilities, the connection felt instant — why not make Gemini the living soul of a text adventure game? The DnD genre turned out to be the perfect vessel: a format practically tailor-made for an AI-powered storytelling engine.

What It Does

Loreweaver is an AI-driven open-world text RPG where Gemini 3 Flash serves as a living Game Master. Players explore, converse with NPCs, and engage in D&D 5e combat — all dynamically narrated by Gemini. NPCs have persistent, evolving memories across sessions. The game generates scene illustrations at key narrative moments and presents everything in a visual novel style interface with typewriter narration and character-by-character streaming.

How I Built It

Built with React 19 + TypeScript and FastAPI, with Firestore for persistent state and Gemini 3 Flash as the core engine. The heart is an agentic architecture: each player turn triggers a Gemini session with automatic function calling enabled. The model autonomously decides when to invoke 19 game tools — recall memory, navigate, NPC dialogue, start combat, generate scene images, trigger story events — and when to produce narrative text, chaining up to 24 tool calls per turn. A post-hoc enforcement layer validates every turn against inferred player intent.

NPCs run on a tiered cognitive architecture with context windows as working memory and Firestore-backed semantic memory graphs as long-term memory. When context hits 80% capacity, it auto-graphizes recent messages into graph nodes, giving NPCs persistent memories. A spreading activation algorithm over hierarchical knowledge graphs (1,300+ nodes, 2,500+ edges) retrieves contextually relevant memories before each narration.

Challenges I Ran Into

Building the right "trellis" for Gemini — the architectural framework that lets it grow elegantly. I went through three major iterations: a fully mechanical hardcoded engine, then a synchronous LLM pipeline, and finally the current agentic loop. Each version felt like a failure at the time, but taught me exactly what tools and constraints Gemini needed.

Accomplishments That I'm Proud Of

The NPC memory system — watching an NPC recall a conversation from hours ago and react accordingly feels genuinely magical. Also the 9-phase world extraction pipeline that transforms raw lorebook files into a full knowledge graph with 1,300+ nodes, making world-building nearly automatic.

What I Learned

Just build it — no matter how imperfect. My mechanical v1 taught me what game operations exist; the synchronous v2 taught me what the AI needs to control; and only then could the agentic loop emerge naturally. Sometimes the detour is the fastest route.

What's Next for Loreweaver

Multiplayer support — letting multiple players share the same world with Gemini mediating their interactions. Deeper NPC emotional modeling with disposition-weighted memory recall. And opening up the world extraction pipeline so anyone can drop in a lorebook and have a playable world in minutes.

Share this project:

Updates