Spectral

Inspiration

• ghosts not being able to interact with the world - very sad
• allows dead people to pursue passion for music
• closest thing in the world of living is the theremin - but theremins need a physical body, and you can only control pitch and intensity at the same time

What it does

• single and multiplayer modes - auto detect
• can control many parameters:
  • left hand hand shape toggles between modes/timbre
  • right hand height controls pitch
  • right hand hand shape toggles between single notes and chords, and controls type of chord
  • spacing between hands - volume and reverb
  • eye-tracking - x-coordinate of gaze pointer controls pan
• displays the value of each musical parameter

How we built it

• consulting with the underworld
• eye-tracking using webgazer - an open source JS library for webcam eye-tracking

Challenges we ran into

• first eye-tracking option was not working

Accomplishments that we're proud of

• multiplayer mode - wanted to see what else is possible

What we learned

• it's ok to start with an existing idea, innovations will come afterwards
• it is important to understand the docs of third-party services, and include it in coding agent instructions
• the dead cannot use trackpads, so we need to pre-calibrate eye-tracking

What's next for Spectral

• get feedback from living and dead musicians - theremin players
• capture entire underworld market (total addressable market: 109 billion people)

Built With

• css
• html
• javascript
• mediapipe
• webgazer