PaperOrchestra

Title card
working

Inspiration

We created PaperOrchestra by mixing computer science, social media, and music together in an amusing and playful way. With the vast applications of hand tracking, we wanted to make an application both intriguing and hands-on.

In primary-junior online school classes, PaperOrchestra could be used as a fun and engaging application that acts as a interactive introduction to both the world of music and computer science. With future improvements, PaperOrchestra could be used for general virtual musical practice or notation.

What it does

Simply put, PaperOrchestra is a video call app where users can join the call and play instruments virtually using their hands. Users are able to play the piano, drums, or the maracas (more instruments to come!). The hand tracking technology detects when and where your hands are placed, and depending on their position, will play notes on the piano, tap the snare drum, or shake the maracas.

Challenges we ran into

In the initial designing process, pinpointing how the piano would work on the screen posed somewhat of a challenge. As it is more difficult for the hand tracker to trace the hands when in standard piano hand posture, we decided to have the hands stand up. Afterwards, the snare drum and maracas were easier to ‘create’.

Accomplishments that we're proud of

We can say that we’re proud of PaperOrchestra’s visuals and implementation of the our designs, on top of its use of hand tracking. Although hand tracking was slightly harder to work with, we are proud of its final capabilities and aesthetics.

What we learned

We learned many various skills, such as:

Drawing and creating a patterned background
Animation through coding

What's next for paper ochestra

As its name suggests, we plan to expand PaperOrchestra to create an orchestra!

Add more instruments
Use the metronome to set the tempo, calibrated to everybody in the call
Record and play music files
Draw the instrument out on paper, adjusted to the screen size to ‘play’ on paper