We decided to use this hackathon to try to address contextual computing in the context of home entertainment. With support from Pebble and Emotiiv, we started building a system to monitor the user’s subconscious and use the data to generate and curate recommendations for film, music, and television. The Pebble watch acts as the primary display center and conscious input device for our platform, allowing the user to react to the prompts from the cloud server (Azure) based off the emotional data, harvested through the Emotiv EPOC+. 

Our system starts with the EPOC+. It monitors electrical activity within the brain. This data is processed on a computer (hopefully future iterations of the hardware will allow us to upload directly to the cloud). The computer pushes the relevant data to the cloud. In the cloud we pull data from the user’s media accounts and run it through a service to generate an emotional profile for every song. We run those songs against the data pulled from EPOC+ to create a list of recommendations. The cloud server (Azure) proceeds push this list to the Pebble Smart Watch where the user is given the ability to choose between a variety of options. The same list is used to prime the content for playing on the media consumption device. The user then will select his her choice from the options presented and the cloud server sends the instructions the television or stereo system. Although as of now it may not be fully implemented, compiling and documenting the outcome of each event has a foundational role in our systems ability to adjust to the user.

Our team decided early on that Pebble was the optimal platform for user input and output for the home media consumption. As a team, we think that for a smart watch to actually be part of your daily routine it has to be both unobtrusive and reliable. We spent a fair amount of time developing an application for the Pebble platform to handle all of the active input and output for the cloud application side of our project. The cloud emulator was extremely helpful in this venture. 

We intend to try to integrate our system with as many media services as possible, starting with Spotify and moving out (due to registration requirements and security tokens, implementing Spotify support, while very straightforward and well documented, is a significant time commitment). Currently we are limited to local audio and video files although implementation for Spotify is partially implemented. Unfortunately, Netflix stopped supporting their public API. We spent some time looking for a work around and may eventually be able to build some sort of remote but leveraging their content is much more difficult than Spotify’s. 

We are working on adding elements of machine learning to our algorithm, allowing us to create unique profiles for our user, placing an emphasis on the algorithm’s failures (spikes in the delta value of the Frustration graph generated by EPOC+) to make our recommendations more useful. A lot of what we are trying to do varies significantly between users. One man’s “sad” music may be another’s “happy “ music, not to mention the variation between genre and decade. Tracking reactions and decisions as well as Emotiv’s quantified emotional data gives us an unprecedented opportunity to accurately adjust to a consumer’s individuality and nuances. 

The core elements of this project have a diverse set of applications. EPOC+ can essentially bypass the subject’s consciousness and provide extremely insightful raw data. Using our system to monitor psych patients or pilots to help quantify and thereby diminish risk would be invaluable to society. Even using it to help couples communicate by showing each member his/her partner’s mood (ideally with just a simple glance toward their wrist) would be incredibly socially rewarding. If your partner knew how you felt, arguably more accurately than you did, even before you said anything, much of the tension that plagues relationships in our fast paced lives could be alleviated. The one thing our entire group agrees on is that it would be a waste to limit the technology we have been developing to a single specific field.

Built With

Share this project:

Updates