SONUX

• 

  Logo

• 

  Demo Display

Inspiration

Have you ever wondered or experienced a world without colour? What is life like being unable to see the surroundings, the people and do entertainment? In this case, hearing becomes an essential sense, which provides people with vision disabilities a unique way to experience the world. In addition, with the growing smartphone penetration rate and the forced stay-at-home caused by the pandemic, more people have gradually become isolated from society. As a team, we genuinely believe that social interaction is a crucial factor in achieving better health. Sonux is here to help build both online and offline connectivity, especially for those experiencing blindness and/or visual impairment. Through playing this game, players will not only gain a satisfying acoustic sensory experience but also encourage more people to stay away from the screen and appreciate the scenery of different communities, of diverse cultures and mother nature, leading to better health outcomes, mentally and physically.

What it does

Sonux is a mobile application designed for anyone looking to relax through screen breaks while gaining a whole experience of auditory sensation. The application is user-friendly, in which people with blindness and/or visual impairment will achieve the same experience as other players. Once players launch the game, the phone’s screen will be in a darkened mode, and players can barely see any displays. The only way to navigate and select game options is through swiping, knocking, or simply waving their phones. As players explore around, Sonux will sense their surrounding environment and search for sounds. At distinct locations, Sonux is able to capture different soundtracks; it can be a lyric of a song, a piece of classical music, the sound of nature or a message from someone unknown. Players are, therefore, able to store all the sounds that they encountered during their journey. As users explore more places, travel to further locations, they will discover diverse sounds. The game enables players to expand their maps to other parts of the world and unlock more soundtracks. Furthermore, players are encouraged to leave warm and positive messages that other users will collect. With multiple sounds collected, users could always return to their sound archive, re-listen to all the sounds, and make various combinations of harmonies.

How we built it

Unity Engine Sonux is developed based on the Unity engine to enable the best 3-D acoustic experience. We started by creating a 3-D space with natural objects modelled inside. Each element has its unique sound and lighting effects, guiding the player's journey. Unlike traditional games where navigation is achieved through screen touches, Sonux utilizes hand gestures, device movement and haptic feedback. By creating a 3-D environment for players to explore, Doppler Effects is achieved virtually by varying sound channels and volume levels, maximizing Sonux's acoustic experience.

Natural Language Processing We employed Unity's voice recognition API with a sentiment analysis model to analyze players' appropriateness of message input. After translating the player's voice message into texts, we process the text message using the textBlob API, a pre-trained NLP model, to determine a sentiment score of the message. To factor in the potential inconsistency of the model, messages indicating harmful contents will be filtered and further reviewed by our staff.

Challenges we ran into

One of the biggest challenges we ran into was in our application of Unity. Unity is the most suitable real-time game development platform in our case to build the audio game for mobile devices. However, our group has encountered several debugging problems as we lack previous experience and skill set with this tool. We have gone through multiple tutorials, navigated the platform thoroughly, and self-taught the fundamental skills needed to develop Sonux. Nevertheless, when coding for the game to be able to listen and react to voice instructions, we failed the first few trials. It resulted from Mac laptops not supporting certain sound features. Thus, we switched to a PC laptop and adjusted our code to successfully make the voice recognition function. Recognizing this bug was one of the big moments for the project.

Accomplishments that we're proud of

The biggest accomplishment we believe is that as a team, we have worked well together. With some members from a tech background and another from a media information major, we had very thoughtful and insightful discussions in the project brainstorming stage. In which all members have identified and agreed upon the need for mobile games to be more inclusive to the special player group and be more considerate towards players' health. Through working cohesively, we have turned our idea from sketch to an actual presentable model that is able to include all the features we have drafted.

What we learned

We learned how to use Unity, how the platform works, and the foundation to make mobile games. Members with a tech background have gained a better understanding of the coding process for game development and how to navigate game engines. For the member who does not have previous tech/science knowledge, it was a fascinating experience to see how the games played throughout daily lives come into being. In addition, all members have gained well-rounded perspectives by seeing how different people think and approach problem-solving differently.

What's next for SONUX

Sonux comes alive after evaluating both the practical components and considering the ethical and social responsibility that a mobile game company should take on to achieve long-term sustainability. Sonux hopes to raise more attention and care to the visual disability population while also creating a more healthy lifestyle for the ever-increasing mobile game players. With these goals in mind, Sonux will continue to build a more connected, long-lasting and inclusive player community.

Built With

• finalcut
• unity
• visual-studio