-
-
StreamSense dashboard with live Twitch chat, short chat summaries and keyboard-accessible speech controls.
Inspiration
Fast-moving Twitch chat can be difficult to follow while streaming, especially for blind and low-vision creators. Reading every message aloud can also become overwhelming. StreamSense focuses on a simpler experience: short spoken summaries that help streamers catch common questions and repeated messages while retaining control over audio.
What it does
StreamSense connects to a streamer’s Twitch account and receives real chat messages. It groups repeated messages, detects candidate questions and generates short, rule-based summaries at a configurable interval.
Streamers can hear new summaries through browser text-to-speech, generate a summary immediately, pause or resume speech, repeat the latest summary and stop playback. Voice, rate, pitch and volume are adjustable.
A clearly labelled demo batch sends 28 fictional messages through the same processing pipeline. Demo and Twitch have separate queues and histories, preserving the real Twitch connection.
How we built it
The frontend uses React, TypeScript and Vite. The backend uses Node.js, TypeScript and Express, with Twitch OAuth and EventSub WebSocket for authentication and live chat. Twitch credentials and tokens remain on the backend.
Speech uses the browser SpeechSynthesis API with on-device voices. The interface includes semantic HTML, labelled controls, a skip link and optional keyboard shortcuts.
The public app runs through Cloudflare Workers with a Render backend.
How we used Codex
Codex helped implement and review the frontend, backend, Twitch integration, summary processing, speech controls, tests and deployment configuration. We worked in milestones, inspected existing code before changes, ran checks and recorded failures and limitations.
Challenges and learning
The main challenges were maintaining OAuth session security, recovering from connection failures, keeping demo data separate from real chat and preventing stale speech from creating an audio backlog.
We learned that text-to-speech alone does not establish accessibility. Users also need discoverable controls, predictable keyboard navigation and a manageable relationship between product speech and their screen reader.
What we verified
The latest full automated suite passed 228 tests. The creator separately verified hosted Twitch authorization, incoming messages with an increased counter and audible generated summaries.
Automated tests use controlled substitutes where appropriate; they do not prove native screen-reader or audio usability.
What’s next
We want to test the complete workflow with blind and low-vision streamers, improve control discoverability, evaluate screen-reader audio overlap and expand question grouping.
StreamSense is a working hackathon preview. Its summaries use conservative rules and can miss context or unfamiliar paraphrases. Full accessibility validation and durable session storage remain future work.
Built With
- cloudflare-workers
- codex
- eventsub
- express.js
- node.js
- openai
- react
- render
- twitch
- twitch-oauth
- typescript
- vite
- vitest
- web-speech-api
- websockets
