Balloon

Balloon is a mixed-reality experience where you try to keep a virtual balloon in the air while moving through your real space and colliding with spatial obstacles.

Comment 9

Inspiration

The act of keeping a balloon in the air is truly a cultural phenomenon. From games that take place in living rooms, to the Balloon World Cup, to episodes of Bluey, it's an activity that people take part in all over the world. What if we could recreate this activity, while adding virtual elements that take the game to an entirely new reality?

What it does

Balloon delivers a controllerless mixed reality experience that seamlessly integrates virtual elements into the player's real-world environment. We created a full spatial audio identity for Balloon providing 360 degrees of sound that delights with original music, informs people with dynamic and high quality text-to-speech, directs people to objects of importance even if out of view, and adds realism and reinforces people’s actions in surprisingly natural and delightful ways. The result is a playful and engaging experience that encourages physical movement and interactivity, all within the comfort of the player's own room.

How we built it

  • Remote, US bi-coastal team of 4
  • Project built in Unity 2022.3.5f, using MetaSDK v64 with Plastic CMS Version Control
  • Communication on Discord and Messenger
  • Google Suite used for project management, tasks, copywriting, slide deck, and asset management
  • Blender 3D object design
  • AI image, texture and code generation with RunwayML, ChatGPT, Unity Muse
  • Adobe creative suite for branding and asset design
  • Recorded real-world objects and designed unique content in Pro Tools and Logic Pro
  • Final Cut Pro video editing

Challenges we ran into

  • Early on we experienced delays with mixed reality passthrough layer issues which took a few days for our team to sort out with support from the Devpost technical team and hack community.
  • We needed to develop custom scripts to reorient objects using MRUK sample FindSpawnPosition script

Accomplishments that we're proud of

  • Remote collaboration across multiple time zones
  • Created 16 custom 3d objects with unique art style setting the nostalgic vibe
  • Designed a unique particle system for frames, balloon pops and wind from the hand fan
  • Generated 50 AI art selects adding variety, surprise, and humor to as wall art in the experience
  • Used Unity Muse for AI texture generation of nearly 20 custom art frames and code generation streamlining art and development investments
  • Studio recording and editing of in-game elements like balloon interactions, collisions and musical elements resulted in more than 120 implemented audio assets
  • Composed original music for game menu, in-game stingers and video submission bringing a cohesive musical theme to the app and related marketing
  • Dynamic TTS scripts and character voice Rosie bring narration to the game without audio assets
  • Turning a concept into a fun and memorable experience we can share with others
  • Hearing friends and family enjoy the experience
  • Collective design ideas result in a cohesive and unique experience

What we learned

  • Using MetaSDK’s PoseProviders and ShapeRecognizer scripts is an excellent way to create natural hand gestures that also add delight and an eased learning curve to the game.
  • Passthrough makes it easy to move through and engage in a real room-based experience unlike VR where there can be little movement.
  • Sound design can replace haptic feedback in controllerless experiences resulting in immersive interactions
  • Spatial audio makes it easy to identify virtual objects out of view in MR
  • Dynamic sounds and sonic variety result in a more enjoyable and pleasing sonic experience.
  • wit.ai TTS characters provide flexible and high quality dynamic sound in ways that would be extremely challenging and size prohibitive to create and implement using traditional audio files, concatenation systems and voice talent.

What's next for Balloon

  • Gain feedback from anyone interested in trying Balloon
  • Make improvements and launch on App Lab
  • Share the project and learnings broadly and scope a deeper app investment
  • Explore local and remote multiplayer

Built With

  • c#
  • chatgpt
  • genai
  • metapresenceplatform
  • metasdk
  • midjourney
  • muse
  • runwayml
  • spatialaudio
  • tts
  • unity
  • wit.ai
+ 9 more
Share this project:

Updates