NEAT SMB

Inspiration

The main inspiration for this project came from a video by technical Minecraft YouTuber SethBling, who implemented the NEAT algorithm to play Super Mario World for the SNES. His work demonstrated how neuroevolution could be applied to classic games, motivating us to adapt a similar approach for Super Mario Bros. on the NES.

What It Does

This project implements the NEAT (NeuroEvolution of Augmenting Topologies) machine learning algorithm to train an AI to play Super Mario Bros. The AI takes in an array of inputs representing the 16x14 grid of visible tiles in the game (the player's immediate surroundings) and progressively associates inputs with actions, guided by rewards based on Mario's progress through the level. This reinforcement learning approach allows the AI to evolve and improve its performance over time, ultimately learning to navigate obstacles, defeat enemies, and complete levels.

How We Built It

Emulator & Tools: We used the FCEUX NES emulator, which supports Lua scripting, to interface with the game and extract real-time data (e.g., tile maps, player position, enemies).
NEAT Implementation: Adapted the NEAT algorithm to process game states, generate neural networks, and evaluate fitness based on in-game progress (distance traveled, level completion).
Training Loop: The AI iteratively plays the game, with each generation evolving networks through mutation and crossover, prioritizing high-performing individuals.

Challenges We Ran Into

Lua Scripting: This was our first major experience with Lua, which introduced a learning curve due to its syntax and quirks (e.g., 1-based indexing). However, it was necessary for interacting with FCEUX.
Performance Bottlenecks: Balancing emulation speed with training efficiency required optimizations to avoid excessive runtime.
Game State Parsing: Extracting consistent and meaningful inputs from RAM (e.g., tile data, enemy positions) involved reverse-engineering memory addresses and debugging misread values.

Accomplishments That We're Proud Of

Successfully integrating NEAT with an NES emulator to control Mario in real time.
Achieving measurable progress, with later generations consistently outperforming early ones (e.g., clearing gaps, avoiding enemies).
Building a modular system that could be extended to other NES games or more complex inputs (e.g., sprite-based vision).

What We Learned

Lua & Game Hacking: Gained practical experience with Lua scripting and memory manipulation in emulators.
NEAT Dynamics: Deepened our understanding of neuroevolution, including speciation, fitness functions, and hyperparameter tuning.
Reinforcement Learning: Explored how rewards and penalties shape AI behavior in a dynamic environment.

What's Next for NEAT SMB

Custom NES Emulator in C
- Develop a lightweight, headless emulator from scratch in C for precise control over game state and performance.
- Prioritize deterministic emulation (frame-perfect synchronization) to ensure reproducible training.
- Implement parallelized training support (e.g., via MPI or multithreading) to evaluate hundreds of NEAT instances simultaneously.
Optimizing the NEAT Network
- Architecture Tweaks: Experiment with constrained network topologies (e.g., limiting hidden layers) to reduce overfitting and improve generalization.
- Input Abstraction: Replace raw tile grids with higher-level features (e.g., "distance to next obstacle") to streamline learning.
- Hardware Acceleration: Leverage GPUs (via CUDA/OpenCL) to accelerate neural network evaluations during emulation.
Long-Term Goals
- Full Game Completion: Scale training to conquer all levels of Super Mario Bros., including warp zones and glitches.
- Generalization to Other Games: Adapt the framework for Mario Kart or Asteroids by modularizing input/output pipelines.