We wanted to built a fun, lighthearted project that is still useful for ML researchers, and settled on a fun little game that allows a martian rover to navigate its terrain! One of our members, Michael, has worked on training RL agents using OpenAI's gym framework during his research and past internships, and wanted to create another open-source environment. The other member, Michelle, just thought it was cool!

What it does

Our environment allows a human or an AI to control a rover on the surface of Mars whose objective is to navigate toward its goal location. The environment also allows the user to specify different "forces" that act on the rover, making the environment dynamic (i.e. not stationary), which is a challenge in modern RL research. Because of this challenge, we hope that it will serve as a useful benchmark for future research.

How we built it

We used python for the whole project. The game itself was built using pygame, and the framework used by the AI to control the game is OpenAI Gym.

Challenges we ran into

Figuring out how each of the environment's components worked together was challenging. We also had a lot of difficulty figuring our the kinematics of the rover and integrating that with pygame's framework. We ran in to issues of timing as well; We thought we would be able to finish more tasks (like benchmarking the environment using a modern RL algorithm like PPO), but we ended up not having the time.

Accomplishments that we're proud of

We are happy to have made a fully-functional game that integrated well with Gym! Michelle enjoyed learning more about reinforcement learning and the frameworks that surround it. Michael enjoyed learning about different features of pygame.

What's next for OpenAI Gym: Martian Rover Dynamic Environment

We want to randomize the landscape and made it more challenging for the rover to navigate. We also want to add more actions to the rover to allow it to navigate a more complex environment. For example, the rover might be able to lift its front wheels in the air to crawl over a boulder.

We want to test the game using modern RL algorithms as a proof of concept, and eventually submit a PR to the Gym Github repo. We also want to create a web-based demo of our game.

Built With

Share this project: