Shorter Audio Listening Experience Supplement

We are quickly approaching a world where AR-capable experiences will be commonplace. Glasses, windshields, goggles, etc. will provide surfaces for virtual content overlay. In this mixed world of real and virtual, we will of course continue to listen to music while performing our daily tasks. Imagine a world where you can mount a virtual picture frame on your wall that features beautifully rendered album art for the song currently playing on Spotify. If you find yourself entranced by the beauty of a piece of virtual art, you could bring it into reality by ordering a full print (or even a vinyl) with a simple interaction. You can enhance the experience further by adding more virtual picture frames that can be automatically filled by images and videos that align with the artist's vision for the currently playing song. Engage as much or as little as you like.

This project is a passive, optionally interactive visual experience designed to enhance the traditional passive listening experience. This version was specifically designed for consuming music from the Wayne Shorter asset collection.



Use Case

Create an immersive visual experience for the consumer to enjoy while listening to music using an AR or VR headset. The musical experience is enhanced when the listener gets to feel the change in the sentiment expressed through light changes, image style transfers, and other artistically choreographed experiences. We are also exploring the ways in which the artist themselves have classified the music via interviews or articles and using mood tagging to help determine how the visual content could and should change as the content library is being explored.


Deep Learning

Artist Exploration

Wayne Shorter is a jazz saxophonist and composer. His music does not consist of lyrics and is classified as jazz, so there is not necessarily a consistent rhythm or tempo to the music. The audio polyrhythms attempt to convey emotion which would usually be described in words. This style leaves the listeners consumption up for interpretation and by enhancing the experience with visual stimuli we can help the listener explore the sentiment the composer intends to convey.

Sponsor Technology Integration

  • Blue Note Assets via Cloudinary
  • Cloudinary Style Transfer

Built With

Share this project: