Inspiration
I've always been interested in artificial intelligence, and how we can use it to learn things humans still haven't been able to figure out on their own. Inspired by the success of projects like AlphaGo, I wanted to see if I could make my own version for a little-known game that I've spent many hours on.
What it does
A PPO reinforcement learning agent that autonomously plays Dungeon of the Endless. A C# mod hooks into the Unity runtime via reflection to extract game state and inject actions over TCP. A Python agent learns strategy (building, exploration, combat positioning, escape timing) through self-play — no vision, no scripted rules.
How we built it
- Reverse-engineered the game with dnSpy to map internal classes
- Built a BepInEx mod (C#/.NET 3.5) that extracts 20+ state types and handles 19 action commands
- Connected it to Python via length-prefixed JSON over raw TCP sockets
- Wrapped the game as a Gymnasium environment with NetworkX graph reasoning
- Implemented a hierarchical PPO agent (strategic brain + combat micro-controller + escape controller) in - PyTorch
- Used Kiro IDE's spec-driven development to manage the full architecture ## Challenges we ran into
- Game runs Unity 5.0.3 / .NET 3.5 — no modern libraries available (no ZeroMQ, no Newtonsoft, no async/await)
- BepInEx standard builds crash on this Mono version; required a patched fork
- Unity's single-instance mutex blocks parallel training environments
- Reward tuning: negative rewards caused policy collapse into "do nothing" — had to make positive signals louder
- Heroes have auto-targeting combat so positioning IS the tactical lever, not commands ## Accomplishments that we're proud of
- Full bidirectional game control with zero computer vision
- The agent successfully learns to open doors, build modules, and escape floors through pure trial and error
- Built the entire C# mod + Python RL pipeline + training infrastructure in one week ## What we learned
- Reward shaping matters more than network architecture in early RL training
- .NET 3.5 is brutally limiting — hand-rolling JSON serialization and TCP framing from scratch
- Hierarchical action spaces (pick what to do, then parameterize how) make intractable action spaces learnable -A working heuristic baseline is invaluable for debugging the RL system — if the heuristic can't do it, the env is broken ## What's next for PPO Agent for Dungeon of the Endless
- Due to limited time, the agent still struggles to pass the first level. Eventually, I want this model to learn to beat the entire game consistently.
- The whole point of making AI's to learn games is to discover new strategies. Once the agent has perfected the game, I'd like to implement a sort of "coach" mode where it can give you tips as you play. It can prevent you from making blunders, and explain how to improve.
Log in or sign up for Devpost to join the conversation.