SpicyTracks

Kitchen music-making experience accessible to all

Comment 2

Inspiration

Our team came together through a shared passion for music. While many people love music, it remains inaccessible to some—whether due to a lack of musical training or mobility limitations. Traditional instruments can be challenging or even impossible for those individuals to play, making co-music-making even more difficult. To address this, we designed a new music-making experience inspired by the kitchen—a familiar space of creativity and learning, often used in cognitive training. This approach makes music more inclusive and accessible to a wider audience. And it is fun and beneficial for mental well-being!

Here is a quick sizzle reel of our project!

What it does

Playing Music with Kitchen Items Grab a pan, or push the toaster—every kitchen item becomes an instrument. With carefully crafted sound design, everyday actions transform into a symphony.

Creating Music with Your Eyes

A simple glance at glasses and spices brings melodies to life, making music creation effortless and intuitive.

Recording, Sharing, and Collaborating

Hit record, loop your sounds, and seamlessly share with friends to co-create a song—anytime, anywhere.

Harmonious Audio and Visual Design

All audio and visual elements were crafted to give a sense of calm and joy, inviting the user to relax and have fun. Also, audio pieces were designed in a pentatonic scale so users could have fun creating whatever song they wanted and always play correct notes.

Other real-world applications

Our musical kitchen concept offers potential benefits for individuals with cognitive impairments. People living with conditions such as dementia or Alzheimer's disease often struggle with memory recall, task sequencing, and maintaining emotional regulation during daily activities (https://pmc.ncbi.nlm.nih.gov/articles/PMC4362400/). Since kitchen environments are commonly used in cognitive rehabilitation, our musical kitchen could offer a safe space to:

  • Stimulate memory recall through familiar melodies and rhythms
  • Enhance mood and reduce anxiety during meal preparation
  • Provide sensory cues to assist with task sequencing and completion
  • Encourage movement and physical activity

How we built it

We began with ShapesXR to quickly prototype a kitchen in a mixed-reality space and test it on Meta Quest. Everyone took turns "playing" in this prototype kitchen, guaranteeing it was comfortable among users.

Our 3D artist and sound designer/musician collaborated, leveraging the designer's synesthesia, to ensure a cohesive audio/visual theme. A distinct musical character was created to convey a mix of warmth, coziness, and fun while maintaining an emotional yet understated tone. The resulting soundscape was designed to be relaxing yet engaging, exciting without being overly action-oriented, and perfectly aligned with the project's overall atmosphere. Accessibility was prioritized by designing the audio system around the pentatonic scale, allowing anyone, regardless of musical expertise, to enjoy playing the kitchen instruments. This approach also enabled seamless integration with pre-recorded background tracks and the layering of multiple players' recorded inputs, resulting in a harmonious and cohesive soundscape. The UX/UI designer mapped the user journey and designed the interface and interactions. Developers then integrated all components into Meta Quest. Through multiple user tests, we refined the experience for a seamless and immersive music-making environment.

How to play with SpicyTracks

-Gaze-controlled instruments are the boxes, spice rack, and glasses. -Hand-controlled instruments are the frying pans and toaster. -The microwave enables you to start and stop your current recording, or trash it and start over, enabling a social element to collaborate on a song. -The microwave also provides a background track that has 3 levels of energetic intensity to inspire musical composition and play.

Challenges we ran into

We ran into a few unexpected devices and build issues at the beginning - and thanks to mentors’ help, we resolved them quickly. It also took time for us to carefully design a harmonious blend of sounds for kitchen items, ensuring a natural and engaging musical experience.

The audio director addressed several challenges to create a cohesive and engaging experience. A unified sound-music language was developed to harmonize with the visual design. The 3D artist crafted a color palette and shapes while listening to the sounds, and additional inspiration was drawn from the 3D models created by the artist to design complementary audio. To ensure consistency, real-life instrument sound libraries were utilized for both the music tracks and custom kitchen-inspired instruments, which were subtly modified to reflect their associated kitchen objects.

The hardware limitations of the Quest speakers, which struggle with low-range frequencies, were addressed by designing music and sounds without a low-register range, ensuring the audio felt complete and balanced despite these constraints.

Another challenge on the development side was harmonizing the musical elements with the physical interactions to ensure the experience felt intuitive and engaging. In addition, achieving smooth sound transitions for grabbable items, such as adjusting the audio dynamically based on the shaking motion presented another layer of complexity.

Accomplishments that we're proud of

Our visuals and sounds communicate in a unified language, creating a cohesive and harmonious experience. We are really proud of the assets we made and how seamlessly they were integrated into XR. We successfully implemented head-tracking for music playing, and a vision of eye-tracking—an innovative and accessible approach that broadens musical expression.

What we learned

A positive and fun team vibe makes everyone happy to collaborate!

What's next for SpicyTracks

-This app will continue expanding the music-making experience with more rooms, eventually covering all rooms of a house. -Diverse new input methods will allow users to use physical objects in their real space to generate sounds, maybe using ML to identify what is in their environment. -Player collaboration will be enhanced by allowing users to record their songs on different dishes cooked in the microwave, which can be shared and collected both synchronously and worldwide.

Built With

Share this project:

Updates