Greater Toronto Area 6ix

Home Screen
Segment of the Game
Segment of the Game - Charged Up!
One Branch of the Game Ending
Right Arm Signal
Stop Arm Signal
Left Arm Signal

Inspiration

As driving becomes increasingly necessary, understanding the rules of the road is essential for ensuring both personal and public safety. At the same time, as green initiatives continue to be promoted, more people are choosing cycling as an alternative method of commuting.

Because of this increase in cycling, there has also been a rise in cycling-related incidents caused by miscommunication between drivers and cyclists. One barrier we identified is that many drivers are unfamiliar with the hand signals used by cyclists.

Our goal is to help new drivers become more familiar with these signals through gamification, allowing them to learn and retain the meanings of cycling hand signals more effectively.

What It Does

We created a two-player, real-time interactive embedded system in which each player controls a separate thread of the system. One player takes on the role of the cyclist and uses our real-time computer vision camera, which applies AI to detect specific arm gestures commonly used in cycling, such as turning left, turning right, and stopping.

The other player controls the car using a custom-built steering wheel equipped with gyroscopic tracking. The system measures the wheel’s yaw, pitch, and roll to determine the direction in which the player is attempting to drive.

The cyclist’s goal is to survive for as long as possible by avoiding the car and additional obstacles that appear throughout the game. The game ends when the car catches up to the cyclist or when the car crashes and is eliminated from the game.

How We Built It

Our real-time AI embedded system is built on a Raspberry Pi 5 and powered by the QNX real-time operating system. We use I²C and other standardized communication protocols to connect the system to its sensors and cameras.

The collected data is sent to a multithreaded C backend. This backend communicates through a network socket, which continuously sends data to the React.js frontend running in the browser. Additional threads handle specific tasks, such as determining whether the steering wheel has been turned. These threads expose their processed data through public API endpoints that the frontend can access and use to update the game in real time.