Shorter Audio Listening Experience Supplement

This project is a supplemental visual experience specifically designed for consuming music from the Wayne Shorter audio collection.



Use Case

Create an immersive visual experience for the consumer to enjoy while listening to music using an AR or VR headset. The musical experience is enhanced when the listener gets to feel the change in the sentiment expressed through light change or image style transfer. We are also exploring the ways in which the artist themselves have classified the music via, interviews or articles and using mood tagging to help determine how the visual content could and should change as the content library is being explored.


Augmented Reality Experience

Virtual Reality Experience

Deep Learning

Artist Exploration

Wayne Shorter is a jazz saxophonist and composer. His music does not consist of lyrics and is classified as jazz, so there is not necessarily a consistent rhythm or tempo to the music. The audio polyrhythms attempt to convey emotion which would usually be described in words. This style leaves the listeners consumption up for interpretation and by enhancing the experience with visual stimuli we can help the listener explore the sentiment the composer intends to convey. By pulling in data from Rovi and interview we know what the composers intent when writing the music or playing the music was, and we want to create an experience which parallels that way of thinking about the song.

Sponsor Technology Integration

  • Mirosoft HoloLens
  • 7digital library
  • Cloudinary Style Transfer
  • TiVo/ Rovi Metadata

Built With

Share this project: