With Soundscape, we wish to share the therapeutic impact that music and drawing has had on our own every day lives. This project is inspired by a number of personal experiences such as practicing calligraphy, creating and listening to music. One of our sources is the reddit r/place page in 2017, a social experiment that revealed a phenomenon in collective efforts to create impressive pieces of art. From our experiences outdoors, we believe that the ambience and soundscape of an environment directly impacts our state of mind.

What it does

As an art installation, the screen/projector offers a simple interface, one line that moves at a constant rate with the music. The user draws on the canvas with intuitive hand gestures in front of the optical sensor. When the line moves across the resulting art piece, the tone of the music reacts in real time. Over the next few minutes, the artwork slowly fades away to make room for new creations.

How I built it

We used a Leap Motion sensor as our main control interface. We built a visual and data processing network using TouchDesigner, running on one laptop. Visual data is processed and mapped to musical parameters, which is sent in OSC packets over UDP to another computer in charge of MIDI generation and sound synthesis. Upon receiving real-time processed parameters, MaxMSP running on the second laptop generates the MIDI messages which are internally routed to Ableton Live for sound synthesis.

Challenges I ran into

In this hackathon, we chose to push our boundaries and use technologies that we have never used before, such as TouchDesigner and the Leap Motion. We ran into many dead ends during this process, and switched between using Javascript and C++ backends before switching to an entirely new creative technology stack. After these initial setbacks, staying up all night was certainly another one.

Accomplishments that I'm proud of

We are proud to be able to present a fully functional product while developing new skills in an unfamiliar process of development. Furthermore, being able to creatively express ourselves through our work.

What I learned

We learned how to drink lots of red bull and coffee. On the more technical end, we learned how to use TouchDesigner, haptic sensors, as well as fundamentals of creative visual development. We also furthered our skills in user-centric thinking and teamwork.

What's next for Soundscape

Stay tuned. ;)

Built With

Share this project: