-
-
Manage Twitch account connections and community memory data from the Stream Memory settings dashboard.
-
AI-generated memory proposals with supporting chat evidence, confidence scores, and approval controls.
-
Approved community memories organized into a searchable timeline with supporting context.
-
Stream Memory dashboard showing community statistics, Twitch connection status, and the workspace overview.
-
Live Twitch chat captures community conversations, inside jokes, and memorable moments.
-
Viewers use !lore to ask questions and receive answers grounded in approved community memories.
Inspiration
Every Twitch community has its own personality. Over time, viewers create inside jokes, recurring characters, memorable incidents, and traditions that make a channel feel like a community.
But Twitch chat moves incredibly fast. A hilarious moment from yesterday can disappear under thousands of new messages. New viewers miss the context behind old jokes, and even longtime viewers gradually forget the stories that shaped their community.
We wanted to solve a simple problem: What if a Twitch community could remember its own history?
That idea led to Stream Memory, an AI-powered memory layer that transforms fleeting chat conversations into a persistent, searchable community history while keeping the streamer in control of what gets remembered.
What it does
Stream Memory turns meaningful Twitch chat moments into streamer-approved community lore.
It provides two main experiences:
For streamers:
- Connect their Twitch channel through OAuth.
- Capture recent chat conversations using
!remember <topic>. - Receive structured memory suggestions generated from chat context.
- Review, edit, approve, or reject suggestions through a dedicated Memory Inbox.
- Explore approved memories through a searchable Community Lore timeline.
For viewers:
- Continue chatting naturally without changing how they use Twitch.
- Ask questions about community history using
!lore <question>. - Receive answers directly in Twitch chat, grounded in approved community memories.
For example, if a community adopts a blue dragon named Byte as its mascot, the streamer can capture and approve that moment. Later, viewers can ask !lore who is Byte? and rediscover the story.
Our central principle is that AI can suggest what is worth remembering, but the streamer decides what becomes permanent lore.
How we built it
We built Stream Memory using OpenAI Codex as our AI coding agent throughout the development process. Codex helped us plan the architecture, implement features, debug integrations, write tests, and iteratively improve the application.
Our technology stack includes:
- Python and FastAPI: Backend services, API endpoints, chat processing, and memory workflows.
- React and Vite: Interactive streamer dashboard with Memory Inbox, Community Lore, and workspace settings.
- Supabase and PostgreSQL: Persistent storage for communities, memory proposals, approved memories, and supporting chat evidence.
- Twitch OAuth 2.0: Secure broadcaster authentication and channel connection.
- Twitch EventSub WebSockets: Real-time delivery of channel chat messages.
- Twitch Helix API: Sending generated lore responses back into Twitch chat.
- OpenAI Responses API: Integration point for AI-assisted memory extraction and response generation.
The application follows a simple workflow:
- Twitch EventSub delivers live chat messages to our FastAPI backend.
- A broadcaster-specific rolling buffer maintains recent conversation context, capped at 10 minutes or 300 messages.
- The broadcaster triggers
!remember <topic>to capture relevant context. - The extraction provider generates structured memory proposals.
- Proposals are stored as pending memories in Supabase.
- The streamer reviews and approves them through the React dashboard.
- Approved memories become searchable through community-scoped retrieval.
- Viewers use
!loreto retrieve relevant memories, and responses are sent through the Twitch Helix API.
We also implemented deterministic mock providers for development and testing, allowing us to validate the memory workflow without depending on paid AI API calls.
Challenges we ran into
Building an application that connects real-time chat, AI-assisted processing, persistent storage, and Twitch APIs introduced several challenges.
1. Real-time Twitch integration
Getting Twitch EventSub WebSockets working reliably required handling subscription creation, authentication, connection lifecycle events, and reconnect behaviour. We also had to ensure that chat messages were correctly parsed and associated with the right broadcaster.
2. Making memory extraction reliable
We did not want every chat message to become a memory. The system needed to capture relevant context, distinguish meaningful moments from ordinary conversation, and preserve supporting evidence. We developed a structured proposal workflow rather than directly converting messages into permanent memories.
3. Keeping AI accountable
We wanted to prevent AI-generated responses from inventing community history. This led us to implement streamer approval and approved-memory-only retrieval, ensuring that viewers receive answers grounded in community-approved information.
4. Database and integration debugging
Connecting the backend to Supabase involved resolving schema mismatches, aligning memory source fields, and ensuring that pending and approved memories were handled correctly. We also worked through Twitch chat permissions and Helix API response handling.
5. Testing the complete workflow
Individual components working independently did not guarantee that the entire experience worked. We had to test the complete journey, from receiving Twitch messages and capturing memories to approving them and successfully sending !lore responses back into live chat.
Using Codex as our development partner helped us investigate these issues, iterate quickly, and improve our testing and debugging workflow.
Accomplishments that we're proud of
- Built a working Twitch integration using OAuth, EventSub WebSockets, and the Helix API.
- Implemented broadcaster-specific rolling chat buffers and the
!remembercommand. - Developed a structured memory proposal and moderation workflow.
- Built a React dashboard with Memory Inbox and Community Lore views.
- Implemented persistent memory storage with supporting source evidence using Supabase.
- Ensured that only approved memories are available for community lore retrieval.
- Successfully tested the
!lorecommand and verified that responses appear in live Twitch chat. - Created deterministic mock providers to develop and test the application without requiring paid AI API usage.
Most importantly, we are proud of turning an abstract idea into a working end-to-end experience where a Twitch community can capture, approve, preserve, and rediscover its own stories.
What we learned
This project taught us that building useful AI applications is about much more than connecting an API to a model.
We learned how to integrate real-time event-driven systems, work with Twitch's authentication and chat APIs, design persistent data models, and build workflows where AI suggestions are subject to human approval.
We also learned the importance of grounding AI responses in reliable data. In a community product, remembering something incorrectly can be worse than not remembering it at all.
Working with Codex showed us how an AI coding agent can accelerate software development when paired with clear requirements, iterative implementation, debugging, and proper testing.
Above all, we learned that sometimes the most meaningful application of AI is not generating something new, but helping people preserve something they already created together.
What's next for Stream Memory
We see Stream Memory evolving into a complete memory system for online communities.
Our next goals include:
- Improving memory extraction and retrieval relevance using richer AI-powered processing.
- Introducing memory categories, relationships, and timelines.
- Automatically identifying recurring characters, inside jokes, and community milestones.
- Improving duplicate detection and memory consolidation.
- Building richer community analytics and memory exploration tools.
- Strengthening EventSub reconnection and reliability.
- Exploring support for additional streaming platforms and larger communities.
Our long-term vision is simple:
Every community has a story. Stream Memory makes sure it never gets lost.
Built With
- artificial-intelligence
- css
- fastapi
- generative-ai
- html
- human-in-the-loop
- javascript
- large-language-models
- node.js
- openai-api
- openai-codex
- postgresql
- pydantic
- pytest
- python
- react
- real-time-applications
- rest-api
- supabase
- twitch
- twitch-eventsub
- twitch-helix-api
- twitch-oauth
- vite
- websockets