Inspiration
Mario is ubiquitous around the world: everyone knows the red and blue plumber, one of the main mascots of Nintendo. However, despite how universal Mario may be, not everyone is entirely "able" to be a part of the Super Mario experience. Playing video games is a choice a person has to make, but there are thousands of people that are physically unable to manipulate a normal controller to play any type of video game. Our team decided to address this limitation by getting rid of wonky controllers and confusing inputs and instead allow you to play Mario with only one thing: your face.
What it does
Super Neuro-Mario is a human-computer interface that allows user's to map facial expressions to commands that control Mario in a 2-D space.
How we built it
Using the OpenBCI Ganglion, we are able to pick up electrical signals from the motor neurons that activate in the user's face muscles when they make certain facial expressions. We recorded data from four different electrodes. We placed two electrodes on the forehead to detect if the eyebrows were being raised, one electrode was placed on the left cheek and another electrode was placed on the right cheek. If the eyebrow raise signal was detected Mario would jump, if the left cheek or right cheek signal was detected Mario would move to the left or to the right accordingly.
We collected and plotted the EMG/EEG data coming from the Ganglion using a Python visualizer and started performing experiments to determine the optimal electrode placement that would give us reliable signals. After determining which face muscles we were going to use we started developing an algorithm to filter out noise and finding valid muscle readings over a period of 100 ms. After processing the signals, we send keyboards commands to a Super Mario emulator depending on which facial expression is detected. Interpreting data from the muscles was a serious challenge. Since the electrical signals coming from the muscles are in the order of 100s of microVolts, differentiating noise from valid signals was really difficult.
What's next for Super Neuro-Mario
The development of our Super Neuro-Mario project is one of the first steps to accomplish our goal to allow everyone to interact with computers. The basic structure of our facial expression recognition can be use by quadriplegics to interact with computers.
Log in or sign up for Devpost to join the conversation.