Hands on the keys

session page

Inspiration

Practicing music often involves digital sheets and recording software that require manually turning pages and turning knobs during a session. Taking hands off of my instrument disrupts the practice flow when every beat counts. Despite many chrome extensions and tools trying to free musician’s hands, I believe AI’s computer use will be the interface to stitch all of these tools together so musicians can focus on their session.

Built on top of Waveform Playlist and using patterns from using Logic Pro

What it does

Ask the agent to control the drum, metronome, and piano tracks
Record with Browser Audio

How we built it

Use code mode to allow the agent to clone the current session data and edit the instruments, notation, and rhythm in one turn.

Challenges we ran into

Exploring use cases that justifies tool calls executed browser side when I am used to server propagating states over web sockets.
ChatGPT voice dispatches subagent so its hard to stay within the same session of WebMCP