PleiadesRacingSim

A 2d, arcade style racer, featuring and a continuous ray casting AI competitor trained using Tensor flow hosted on a raspberry PI, which audio powered by Eleven Labs.

Comment

Inspiration

Both Josh and Reid are members of the formula racing team, and Reid personally owns his own racing sim for racing games. Reid and Kate are both CS and Cybersecurity majors who have had lots of interest in the past in training their own machine learning algorithms, due to the affect their pattern recognition abilities might have on the Cyber Security space. These common interests welded together into a fun racing simulation.

What it does

The project is composed of three main parts, the racing game, the machine learning system, and the hardware and network systems that connect and display them. The game records telemetry data of the player character, which is used to train an AI model to control an AI car, which plays against other players. This allows the dev to easily create bots to compete against the user, very quickly. About 5 minutes of playtime data is needed for a decent model, and the training takes about 2 minutes to run.

How we built it

We created the 3 separate system independently, with the understanding that they would have standard inputs and outputs to connect them. The game was built in JavaScript, using phaser.js and vite.js. The game was built so that all relative data, position, rotation, ray distance to wall, etc. was put into a json file after being discovered through ray casting. During this development the training program was being written using pytorch, which knew it needed to accept a json file as telemetry and create a .pth model that would be called by the game to control an AI car. To connect these two systems, websocket were created that worked to communicate between a number of browser calling requests for the AI. Then this whole system needed to be placed onto an Raspberry Pi 5, and assigned a public domain, so that any browser could connect and play the game.

Challenges we ran into

At K-State, port forwarding is not an option, so running a public server off of the Raspberry Pi 5 was much more complicated than usual, resulting in us using Cloudflare Tunnels to connect to a dynamic domain without having to mess with router config. Figuring out how to use Docker was a challenge as compile time ranged between 30 - 40 min as a result of many libraries being downloaded. When attempting to use a remote ARM server to help build the docker image faster, it was found that it couldn't be done because K-State doesn't support external ssh connections. We also tried to get the .tech domain staryourengines.tech but we couldn't in time because of promo code issues. We substituted it for a vaild domain we had previously used in the last hack k-state we attended (https://chiral-practi.study).

Accomplishments that we're proud of

We are proud of getting the AI to train off the game data, obtaining the data through ray tracing and adapting car action according to data received through ray tracing, getting Docker to compile, hosting from the Arm Raspberry Pi, incorporating the Eleven Labs voice features into the game, and more.

What we learned

We learned a lot about the Phaser.js project in Javascript, how certain machine learning algorithms function, and how processes can communicate with each other, especially through the use of a web socket.

What's next for PleiadesRacingSim

Improving the training model would be a natural next step, as the current model can become confused and turns back on itself. Another model type may be something like a reinforcement model, which would take a much longer period of time to get functional, but would have a higher ceiling for the computer's skill. The addition of more player and AI cars onto the grid would make for a more interesting game, with a higher level of complexity. This would require more advanced AIs to deal with the rapidly changing environment.

Built With

Share this project:

Updates