pulse choir

Inspiration

Live streams often communicate through sound, visuals, or spoken interaction. That can make a musical moment harder to follow or join for viewers who are deaf or hard of hearing, blind or have low vision, or nonspeaking. We wanted chat participation to work through a simple text command and to make the resulting music visible and readable as well as audible.

What it does

Pulse Choir turns a Twitch channel's live chat into a shared, beat-aligned instrument. A streamer connects Twitch, chooses the tempo, key, scale, octave, sound level, and who may participate, then adds a private Pulse Choir Browser Source to OBS. Viewers type !pulse 1 through !pulse 7 to contribute notes or !pulse clap for a rhythm tap. The service validates real Twitch chat events, applies participation and rate limits, maps accepted notes into the selected scale, and schedules them on a shared beat. The browser source visualizes the composition and synthesizes original tones.

The dashboard shows the connection and session state, room settings, accepted contribution count, and recent notes. Streamers can pause or resume the choir, mute sound, change settings, and create or rotate the private OBS source link.

The separate accessibility companion presents channel state, tempo and key, a four-beat position, and recent accepted notes as text. It offers adjustable screen-reader announcement pace and detail, pause/resume announcements, text sizing, high contrast, and reduced motion. Nonspeaking viewers can participate through ordinary text chat without voice input. These features are product design and implementation; they have not been validated through formal disabled-user testing.

How we built it

We used OpenAI Codex to help research Twitch's developer interfaces, plan the implementation, write and review the application, and prepare the project documentation. The application uses Node.js and JavaScript on the server, Twitch OAuth for broadcaster authorization, Twitch EventSub chat events for real viewer commands, WebSockets to update the overlay and companion, and a browser-based interface. OBS loads the private overlay as a Browser Source. Web Audio synthesizes the notes locally. Pulse Choir has no runtime AI, model API, or AI-generated music.

What we learned

We learned that a live interaction needs more than a chat command: it must validate the broadcaster and participation rule, handle rapid messages, schedule accepted notes consistently, and communicate the same state across the dashboard, OBS, and companion. We also learned to treat accessibility as part of the interaction itself, with readable text, configurable announcements, and text-based participation.

Challenges

The central challenge was coordinating asynchronous Twitch chat events with a shared musical beat and keeping the dashboard, OBS visualizer, and text companion synchronized. Browser audio also requires an explicit user gesture in the source. OAuth tokens and the private overlay link needed different protections, while the companion had to expose useful music state without revealing viewer identities or raw chat messages.

Built With

• css
• html
• javascript
• node.js
• obs-browser-source
• openai-codex
• render
• twitch
• twitch-eventsub
• twitch-oauth
• web-audio-api
• websockets