We wanted to spread awareness of mental health issues and provide a better understanding of the problems that are faced by sufferers, including those who suffer as a result of harassment. We wanted to recreate the experience of having to deal with anxiety and depression by making use of the technology available to us.
What it does
The interactive experience uses a MySQL database to send heart rate data from an iOS app to a web page using a PHP API. The heart rate is then called from a Unity script which changes the user experience in real-time.
How we built it
We used Unity and C# as a game engine, and connected the PC app to an iOS app that we made. We made use of an algorithm we found online which allows us to measure the user's heart rate in real-time using a technique involving the torch and camera. The heart rate is sent to a MySQL database, and the data is then sent to a web page using our own PHP API. Changes in heart rate are automatically detected by Unity, where a script keeps a constant connection between the interactive experience and the database.
Challenges we ran into
Initially, our plan was to use a MUSE headband - which unfortunately was not available to us. We also intended for the experience to be in VR, however we were unable to rent any of the hardware for the project. We adapted our ideas to make them work with the equipment available to us. The most challenging problem that we had was to keep an open connection between the database and Unity, as the script was making a call to the database once each frame (about 60 calls per second). We managed to reduce the number of calls made and to keep the connection open for the duration of the experience.
Accomplishments that we're proud of
We are very proud of being able to put together a good range of functioning and high-level technology in order to create a smooth synergistic experience. Using the user's heart rate to evaluate how the experience should change was challenging but was extremely rewarding to watch. Seeing a user place their finger on an iPhone camera and watch the experience change accordingly was a good feeling. We're also proud of being able to produce the idea to a degree we are happy with despite not being able to access the technology we initially wanted.
What we learned
We learned how to connect a web page to a Unity script and apply changes in real-time according to the data gathered. And we learned that you can measure your heart rate by scanning your fingertip on your phone's camera! We've learned a lot of valuable things like managing API calls per minute, managing frame-rate in Unity, and manipulating an environment at run-time.
What's next for STORM
Discussing with one of the mentors we decided that there are a few future applications of STORM. One would be to refine the experience in order to assist those suffering from mental health issues express themselves to those who have not. Another possible application would be to expand our database in order to collect information based on age, height and weight with the intention of performing some kind of metadata analysis.