Lunar Olympics

Indoor Sports
Outdoor Sports
Workflow

Inspiration

Lunar base won't be a dream in the next 5-10 years. What about Olympics game?

What it does

Allows multiple sports players to participate in the Olympics game. There are two types of game: indoor and outdoor. For the outdoor games player could enroll in 100 meters race, high jump and shooting. The actions of the player could be controlled by using IMU sensors connected to the cell phone via Bluetooth Low Energy. The hosting area will be located in Apollo 11's landing site. User can use hand gesture to switch to indoor sports which will be held inside a Space-X Starship.

How I built it

I prototyped the scenes for both indoor and outdoor scenes, then I programmed on the Arduino Nano Sense 33 for transmitting IMU in real time. I also trained a deep learning model for the realsense camera to recognize hand gestures, for sports switching. The signal is then updated in Firebase realtime database and will be transacted on the mobile side.

Challenges I ran into

The Realsense T265 camera can recognize hand gestures efficiently in real time with surprising accuracy, due to its depth information besides the 2D picture captured. I spent a long time training the model which could recognize 4 categories effectively.

Accomplishments that I'm proud of

I am able to host a future Olympics game on the moon!

What I learned

It's possible to achieve mixed reality experience with a cell phone and a stereo camera like Intel Realsense, as well as external IMU sensors.

What's next for Lunar Olympics

The scenes for indoor sports will be completed, providing more options for the players. Multi-player interaction could also be added. Further the neural network could be retrained to recognize more gestures or allow the user to interact with virtual objects by using fingertips.