Fly & You

GIF

Other video: https://youtu.be/Db8V_mz_c9s

Inspiration

We saw a reconstruction of a male fruit fly's central nervous system (MaleCNS, Berg) and knew we had to do something with it. We determined the best contrast to the fruit fly's native state is being a walk.

What it does

Runs MaleCNS in our 3D environment, where the user must paint the environment to make certain areas more or less appealing, guiding the fly away from danger and towards the goal.

How we built it

Fly & You is written in Rust, using the Bevy game engine. Early prototypes used the reference implementation for inference using MaleCNS, but we quickly found that to be too slow for a real time use on our hardware. We then implemented a custom kernel to run the inference model at roughly 100x real time, a 10x improvement from the reference code, allowing us to simulate the fly as the player makes changes to the environment.

Challenges we ran into

The Fly is not great at following directions or straight paths. This is a shockingly big issue for a game whose center idea is that the fly is it's own being. Puzzles generally require consistency, and nature is anything but. This led us to the implementation of multiple colors of environment paint, alongside a reduced resolution for the fly's visual stimulus, in an attempt to make the game more consistent and tame the fly.

Accomplishments that we're proud of

Cut down ms/simulated second of The Fly by 11x using a custom Metal kernel.

Very happy with the intro/tutorial.

What we learned

The Fly is both shockingly capable and shockingly stupid. The fly will gladly go head first into danger, no matter how dangerous we made the danger look, or how harsh the penalty. However, the fly is also good at learning from the user and how they use environment colors to guide them.