As avid fans of rhythm games such as GuitarHero and osu, we've always wondered whether training one's sense of rhythm through these games could serve as more than just entertainment. After doing some research about the connection between dyslexia and trouble sensing rhythm in music, we came to the conclusion that rhythm games could have potential to help kids with dyslexia or other learning disabilities train their sense of word recognition by allowing them to read "patterns of color" from left to right going down the screen, much like reading a book. This could be especially effective if paired with a data analysis app that allows teachers, instructors, or parents to track a student's progress and understand how best to help his or her student train their reading skills. Source: https://www.elsevier.com/about/press-releases/research-and-journals/dyslexia-linked-to-difficulties-in-perceiving-rhythmic-patterns-in-music
What it does
The rhythm game itself is fairly standard with dots (mapped from beat maps of popular tunes) coming down the screen in columns with the objective being to hit each dot as it moves down onto a "tracking area." As the game is played, a JSON file is produced in real time that records the player's responses in relation to the target notes. Using Microstrategy's advanced analytics, visualizations of the player's performance help track progress and provide immediate feedback about what types of notes the student is hitting or missing (quarter, eighth, etc.) and what columns they have trouble moving between.
How we built it
We created our game using Unity and C#. rPeggio utilizes queues to take in notes from handmade beat maps to display them in sync with music. We used xcode to build a companion app for instructors that displays all feedback results and tabulates each student's progress.
Challenges we ran into
There was a lot of initial confusion on how we would process the data produced in the game. We had trouble implementing Microstrategy into our projects.
Accomplishments that we're proud of
The game looks very clean and visually appealing. Also our beat maps and hit boxes are very precise.
What we learned
We learned a lot of basics simply by workshops and company tutorials/resources.
What's next for rPeggio
We originally planned to put the game into VR with motion controls as well, so that is a possible direction to take. In addition, this student / teacher model can be used to collect data and conduct research on various diseases. We also want to market it to select schools with special education instructors and researchers.