Shorter Audio Listening Experience Supplement
This project is a supplemental visual experience specifically designed for consuming music from the Wayne Shorter audio collection.
Hackathon
When: June 2-3 2018
Team
Use Case
Create an immersive visual experience for the consumer to enjoy while listening to music using an AR or VR headset. The musical experience is enhanced when the listener gets to feel the change in the sentiment expressed through light change or image style transfer. We are also exploring the ways in which the artist themselves have classified the music via, interviews or articles and using mood tagging to help determine how the visual content could and should change as the content library is being explored.
Technology
- Unity
- Cloudinary Style Transfer
- Microsoft Hololens
- Oculus Go
- Blender (3D modeling)
- NYT API
- ROVI music API
Augmented Reality Experience
Virtual Reality Experience
Deep Learning
- Very Deep Convolutional Networks for Large Scale Visual Recognition
- A Neural algorithm of artistic style
- Cloudinary Neural Artwork Style Transfer
Artist Exploration
Wayne Shorter is a jazz saxophonist and composer. His music does not consist of lyrics and is classified as jazz, so there is not necessarily a consistent rhythm or tempo to the music. The audio polyrhythms attempt to convey emotion which would usually be described in words. This style leaves the listeners consumption up for interpretation and by enhancing the experience with visual stimuli we can help the listener explore the sentiment the composer intends to convey. By pulling in data from Rovi and interview we know what the composers intent when writing the music or playing the music was, and we want to create an experience which parallels that way of thinking about the song.
Sponsor Technology Integration
- Mirosoft HoloLens
- 7digital library
- Cloudinary Style Transfer
- TiVo/ Rovi Metadata
Log in or sign up for Devpost to join the conversation.