Whispers of the Realms - An immersive voice-enabled RPG powered by LLMs.

What Inspired Us

Our inspiration stemmed from a shared love of classic text adventure games like Zork and modern role-playing experiences like Dungeons & Dragons. We asked ourselves:

What if NPCs could actually talk back?
What if quests could be discovered naturally, through conversation?

With the rise of LLMs and voice tech, we saw a unique opportunity to reimagine how players interact with digital worlds—not through menus or commands, but through natural, spoken dialogue.

What We Learned

Prompt Engineering is powerful: giving each NPC a distinct voice and personality created a more immersive experience.
LangChain Memory & Vector Stores help keep conversations consistent and grounded in lore.
Speech Recognition added a whole new level of immersion, but required careful error handling.
Modular design allowed us to dynamically add quests and NPCs, making the game endlessly extensible.

How We Built It

Frontend

Built in Streamlit for quick UI and interaction flow.

LLM Backbone

Gemini and Mistral for personality-rich NPC dialogues.

LangChain for chaining prompts, memory, and retrieval.
Voice Interaction - Implemented via SpeechRecognition.
Modular Files: NPCs and quests are loaded from .txt files and processed into embeddings.
Retrieval-Augmented Generation (RAG): Used FAISS to provide NPCs with local knowledge for better context.
Game State: Managed with Streamlit’s session state—tracking inventory, location, quest progress, and conversations.

Challenges Faced

Voice and LLM Integration: Combining real-time voice input with streamed LLM responses introduced latency and synchronization issues.
Context Management: Maintaining a believable conversation history while preventing context bloat was tricky.
Balancing Control and Creativity: Letting NPCs be creative without them going off-lore was a continuous tuning effort.
Dynamic Loading: Ensuring quests and NPCs loaded on-the-fly without breaking session state took multiple iterations.

Click this for the Game