Qualia

Inspiration

Watching "Ready Player One" the first time made me view VR as a superior medium for gaming. However, the current state of VR doesn't align with that vision, as most games rely heavily on pressing buttons, pressing virtual buttons using hand tracking or unnecessarily complex set of unintuitive gestures to memorize, which significantly limits immersion. Another aspect that doesn't feel immersive is floating Menus. Project Qualia is a new and unique approach to both those aspects, locomotion and menus.

What it does

In Qualia, players can move freely without physical controllers by mimicking real-world movements (e.g., jump to jump, swinging hands to walk, punching to attack). Hand-held menus in Qualia mimics using a phone/tablet in real life.

How we built it

We recognize movement patterns using a combination of hand tracking positions, hand gestures, and headset tracking. These patterns trigger in-game intents that control the player.

Challenges we ran into

False positives in patterns recognition.
Defining robust and reliable movement patterns.

Accomplishments that we're proud of

We successfully created a game that allows us to continuously experiment with new implementations through new "champions." This lets us compare data and determine the most effective/enjoyable control methods.

What we learned

Hand tracking and gestures alone are insufficient to fully replace controllers as an input method. Incorporating other methods like headset tracking, and full-body tracking, is necessary to eliminate controllers without compromising gameplay.

What's next for Qualia

Experimentation, experimentation, and more experimentation. Controllers matured over decades: if hand tracking aims to reach that level, we must rethink the entire input paradigm from a fundamentally different perspective.