Inspiration
“I knew that would happen.” Every stream has that moment. Someone predicts a clutch win, an unlucky outcome, or a disastrous decision—and afterward, everyone remembers being right. But what did they predict before it happened? How confident were they? And who makes good calls consistently?
As a data scientist, I saw an opportunity to turn that familiar stream banter into a shared forecasting game. Predicting an outcome is interesting; putting your confidence on record makes it more revealing.
That inspired Called It: Big calls. Clear receipts. Viewers make their calls before the action, then the result gives everyone something to celebrate, debate, and learn from together.
What it does
Called It lets streamers turn an upcoming moment into a confidence challenge. The streamer creates a clear Yes/No question, such as “Will I survive the next attempt?” Viewers submit their outcome and confidence directly through Twitch chat: !call 14 yes 80 means an 80% confidence forecast for Yes in round 14.
Each account has one current forecast per round. Viewers can revise their call while submissions are open, but forecasts lock before the attempt begins. Afterward, the streamer confirms Yes or No, or voids an unclear or interrupted round. Called It calculates transparent Brier-based scores and reveals who made the strongest calls.
For example, an 80% Yes forecast earns 96 points when Yes happens. A 90% Yes forecast earns 19 points when No happens. Confident mistakes have consequences. OBS overlays show the question, locked crowd forecast, and result reveal. Participation uses free forecasting scores, without wagers or Channel Points.
How we built it
I used OpenAI Codex as a development collaborator to translate the idea into application structure, round controls, forecast parsing, scoring logic, and interface components.
The application separates round management, incoming forecasts, outcome confirmation, and scoring. Each forecast belongs to a specific account and round, with the latest valid submission replacing earlier choices before locking. Twitch chat provides the participation interface, letting viewers join without opening another website. The streamer dashboard manages questions, locking, and resolution.
The scoring engine converts each call into a probability of Yes and applies:
Score = 100 × [1 − (p − y)²]
Here, p is the forecast probability of Yes, and y is 1 for a Yes outcome or 0 for No.
The visual design uses purple broadcast graphics, white panels, and an anime streamer character. Synthetic previews are labelled separately from live activity.
Challenges we ran into
The biggest challenge was making each round unambiguous. “Will I win?” needs a defined attempt and a clear ending before anyone can forecast fairly. Timing also matters. Forecasts must lock before the outcome becomes visible, with consistent rules for revisions and late submissions. Another challenge was explaining confidence scoring without turning the stream into a statistics lesson. Concrete score examples make the consequences easier to understand. Finally, the crowd forecast needs careful presentation: it summarizes viewer probabilities and should never be mistaken for a validated game prediction model.
Accomplishments that we're proud of
I’m proud of turning a familiar Twitch interaction into a repeatable participation loop:
Ask → forecast → lock → confirm → reveal.
The combination of confidence, fixed submissions, and transparent scoring gives viewers something more meaningful than a retrospective “I told you so.”
I’m also proud of keeping the streamer workflow simple while giving the result reveal a recognizable broadcast identity.
What we learned
A good forecasting experience starts with a well-defined question. We also learned that confidence matters beyond whether a prediction was right or wrong. Over many rounds, scoring can reveal the difference between careful judgment and repeated overconfidence. Building with Codex reinforced the importance of specifying edge cases early: revised forecasts, deadlines, voided rounds, and outcome confirmation all affect the experience.
What's next for Called It
The next priority is testing with streamers to find questions that are clear, entertaining, and easy to resolve. Future improvements include reusable question templates, multi-round challenges, customizable OBS overlays, and clearer forecasting histories. With enough resolved rounds, calibration views could help viewers explore whether their confidence matches their results. Those views should show sample sizes and avoid strong conclusions from limited activity. The goal is to make watching more participatory: call the moment, commit your confidence, and see who called it.
Built With
- codex
- event-sub
- fastify
- node.js
- obs-studio
- openai
- openai-codex
- react
- sqlite
- twitch
- typescript
- vite
- websockets
