Inspiration
12% of the U.S. population suffers from social anxiety disorder. AiR is an omnipresent companion and a communication trainer who is going to help you manage your anxiety and improve your communication skills.
What It Does?
AiR assures you that there is always a friend in the room with you. You can talk to AiR about anything and everything while they analyze your speech and facial expression to recommend activities to calm your anxiety and improve your communication skills.
How We Built It
AiR is built using Unity for the Oculus Quest Pro. Character Creator 4 is used to create the digital twin, Microsoft Cognitive Services API is used for speech synthesis and Davinci003 for conversation generation, and EmoRoBERTa API from Hugging Face is used for emotion analysis.
Challenges We Ran Into
- We had access to just one Oculus Quest Pro headset for the development and testing of the application, which prevented quicker feature development.
- The instability of OpenAI ChatGPT affected our user experience and made us shift to Davinci003.
- We had to desexualize default Character Creator models while creating the digital double.
- We were aware of the limitations (hallucination and incorrect medical advice) of large language models while developing an application for managing social anxiety and consciously designed around them to ensure safe and engaging experiences.
- We also wanted to augment our sentiment analysis with inferences drawn from neural data from the brain using BCIs. However, the available devices weren’t in the ideal form factor, and the data was not sufficiently high resolution.
Accomplishments that we're proud of
AR/VR/XR technology is blamed for pushing humans into isolation. So we decided to use the problem to build the solution. We are proud of our minimalist UI/UX design. We thoughtfully and painstakingly designed the interface and interactions to ensure a calming and therapeutic experience for the user.
What We learned
Everyone in the team works exceptionally well together, beyond our wildest imagination, because of our highly diverse expertise and experience, curiosity to learn from each other, and respect and accommodate each other’s opinions.
What's next for AiR
We believe that our project addresses an important issue. Beyond the hackathon, we want to focus our efforts on faster speech analysis and conversation generation, improved visual design, and developing a more detailed behavioral cue analysis pipeline to better customize the recommendations to the user. Our entire team is committed to continuing this project beyond the hackathon. Please contact any team member if you're interested in contributing or advising.
Log in or sign up for Devpost to join the conversation.