BUGSKILL

Inspiration

BUGSKILL was inspired by the quirky and quick mini-games of WarioWare, which find humor and challenge in everyday gestures. Our game turns these gestures into a frantic battle against a house fly infestation in mixed reality, inviting players to swat, clap, and trick their way through waves of pesky flies.

What It Does

Set in a dynamically mapped real-world environment, BUGSKILL challenges players to eliminate all flies in sight using their hands and quick reflexes. Players can slap flies against surfaces, clap them out of the air, or employ a whimsical frog hand puppet to snap them up. This immersive experience is enhanced with spatial audio cues and realistic interactions facilitated by Meta’s advanced hand tracking.

How We Built It

Built with Unity and leveraging Meta Quest SDKs, BUGSKILL utilizes a wide range of tools, including Passthrough, Scene API, Depth API, Mixed Reality Utility Kit, and Audio SDK.

Through the use of Scene API and Mixed Reality Utility Toolkit, we were able to procedurally calculate the flight path of the flies and spawn them near specific areas within a space. By adding colliders to the scene volumes, the flies can check for any possible obstacles in their paths and maneuver around physical objects in the space. The Mixed Reality Utility Toolkit also provides a convenient method for generating random points on any vertical or upward-facing surfaces. By utilizing the semantic labels and bounds provided by the scene model, we can ensure that the flies only spawn near windows or doors.

We intentionally designed the game with hand tracking only to evoke the feeling of catching flies in real life. Wide Motion Mode significantly enhances hand tracking accuracy and reduces lag, enabling us to maintain high-paced gameplay without compromising tracking quality. The hand physics capsules also handle collisions with precision, inspiring the implementation of various hand-based mechanics, such as clapping and slapping against surfaces.

We leveraged the capabilities of spatial audio to provide users with a better sense of the flies' locations. Using Meta's Audio SDK, we implemented audio cues to easily track where the flies are relative to the player's head. Additionally, we added the Doppler effect to flies that are moving toward or away to heighten players' sensitivity to targets.

Passthrough is key to the immersive gameplay of BUGSKILL. Players feel safer moving and navigating in Passthrough mode, allowing us to take advantage of a larger play area and provide them with a sense of agency during gameplay. Furthermore, occlusion by the Depth API truly elevates the game to the next level, as flies can easily hide behind physical objects, encouraging players to strategize carefully. By intentionally prompting users to apply everyday hand gestures and interact with physical space within the headset, the game enhances players' spatial awareness and sense of safety with mixed reality technology.

Challenges We Ran Into

Our development process faced several challenges. Merging various development streams into one coherent project required careful coordination and extensive testing. We also had to refine our hand interaction mechanics, ensuring they were responsive and didn't overlap, which was essential given our game's reliance on precise hand tracking. Additionally, confining flies within the gameplay area was tricky. Initially, flies would escape or get trapped when spawned inside scene volumes, prompting us to adjust our spawning logic to keep them visible and accessible.

Accomplishments That We're Proud Of

We successfully integrated occlusion into our custom-written shaders, which significantly enhanced the realism of the game environment by allowing virtual objects to interact more naturally with the real world.

Another major accomplishment is the detailed sound design and implementation of spatial audio. The realistic sound effects, such as the buzzing of flies close to the player’s ears, dramatically enhance the immersion. Thanks to Meta Audio's spatialization capabilities, players can discern the location of a fly—whether it's behind and low to the floor or above and in front—without needing to see it. This precision in audio cues makes players swivel their heads as if a real fly were whizzing by, supporting the icky, visceral reaction players have during gameplay, which has been both a goal and a highlight of the project.

These achievements not only reflect our technical skills and creative vision but also our commitment to delivering a compelling mixed reality experience that feels as real as it is entertaining.

What We Learned

This project not only enhanced our understanding of Meta’s development tools but also highlighted the utility of the Meta Presence Platform in rapidly creating immersive MR experiences. It underscored the importance of frequent testing and helped us identify our team's strengths. We learned the value of iterative design, prioritizing functionality early on, and refining details later, which were crucial in managing the complexities of mixed reality development.

What's Next for BUGSKILL

Looking ahead, we plan to expand the game’s arsenal with more fly-catching tools like swatters and sprays, introduce items that can be earned, such as insect repellent after reaching specific milestones, and add a new category of household pests, challenging players to tackle a variety of infestations. Future updates might also include symbiotic environments and creatures that interact with these pests, enhancing the ecological aspect of the game. We're also exploring competitive multiplayer modes, where players can team up to clear infestations together.