We are a team focused on developing a new tool for piano learners at any level. Through marketing research of piano learners in different regions around the world we obtained the following estimates: while the European piano players are declining, the increasingly dominant user markets are in the US and China. The former has an estimate of over 20 million piano players, and the latter has a piano-learning frenzy rapidly taking over 40 million children alone - that's more than the current Canadian population (The American Association of Piano Teaching?; BBC news)! In addition, there are around 15 million youth or adults striving to make a living in a piano-teaching career (..). However, although there is a plethora of released products related to mastering this particular instrument, most focus on providing visual or auditory tutorials. For instance, the app created by a professor here at the Peabody Institute of Music of the Johns Hopkins University is positively reviewed for its advantage of providing sight reading training. Another app named Piano 3D has been rated a 4.5/5 by 6937 ratings in the Apple App Store for its game-like exercises combining auditory and visual support. The lack of training in tactile sense in piano learning - i.e. the sense of touch - inspired us to create a product specifically geared towards training of fingering, rhythm and musical expression.
What it does
We designed a glove with sensors at the tip of each finger wrapping. When a user plays on the keyboard with the gloves, the sensors would send tactile information (pressure signals) to our database. There are two ways of using the database. The first is to apply a big-data strategy and customize lessons based on the tactile sensation data uploaded. The other is to use a VR glove and play back the stored tactile sensations of any piece onto the users fingertips such that they learn the amount of pressure best to apply to the keyboard.
How we built it
We used soft piezo-electric sensors and a latex glove to make a prototype of glove to record the tactile sense. The prototype is built with an Arduino and a breadboard. As the fingertip pressure is applied to each sensor, the Arduino reads the voltage value acting on the sensor, which is then transferred to the value of pressure. The pressure distribution of every finger is uploaded into our database for future reference. For a demo of replaying the tactile sensation, we used an Arduino to manipulate solenoids to apply a force on the fingertips.
Challenges we ran into
Holding and gauging of the pressure sensors; designing the wiring of the components; reading data from the Arduino port and importing to a file; measuring data using the prototype glove on a real grand piano while shooting high-quality videos simultaneously; writing an algorithm to match the pressure signals for the actual piece of music recorded; 3-D printing the shell of the workstation for the solenoids; activating the solenoid according to the pressure signals desired.
Accomplishments that we're proud of
We did a thorough marketing research and customer discovery, and tested the idea by conducting 40 interviews. We made a working prototype for the measuring glove and used it while playing a J. S. Bach French Suite.
What's next for TouchArt
We look forward to passing the I-corps program for customer discovery. We will update the prototype with thinner and softer sensors to build a glove that will interfere less and less with playing. We will collect data from pianists and students using a working sensor glove. We will develop the algorithm for analyzing the tactile sensation from different users and return advice on the way to improve playing. We will continue to develop the actuator and build a replaying glove that could potentially reproduce the pressure signals back onto the fingers of users.