-
-
dashboard
-
trading screen
-
event listener
SpeedrunFi — Project Overview
Live prediction markets for Twitch speedruns, powered by XRPL and Gemini VLM.
SpeedrunFi is a Chrome extension that turns live Minecraft speedrun streams into real-time prediction markets. Viewers trade YES/NO tokens on the XRP Ledger while a vision model (Gemini) watches the stream and automatically resolves markets when runs complete.
Table of Contents
- Overview
- Architecture
- Components
- Stream → Market Mapping
- XRPL Integration
- Data Flow
- Setup & Run
- API Reference
- Demo Guide
Overview
What It Does
- Watches a Twitch speedrun stream via tab capture
- Detects game events (Nether, Blaze Rods, Stronghold, End, Dragon Kill, Run End) using Gemini VLM
- Creates one prediction market per speedrun attempt
- Trades YES/NO tokens on XRPL Devnet (native DEX)
- Resolves automatically when the run ends (YES if dragon killed, NO otherwise)
- Suggests new market questions via LLM based on stream content
Tech Stack
| Layer | Technology |
|---|---|
| Extension | Chrome Manifest v3, Vanilla JS |
| Node Server | Express, WebSocket (ws) |
| Stream Oracle | FastAPI, Gemini 2.5 Flash |
| Blockchain | XRPL Devnet, xrpl.js |
| VLM | Google Gemini (vision + function calling) |
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ CHROME EXTENSION (Sidebar) │
│ ┌─────────────┐ ┌─────────────────────┐ ┌─────────────────────────┐ │
│ │ Dashboard │ │ Watch (iframe) │ │ Trade │ │
│ │ Portfolio │ │ Stream Watcher UI │ │ XRPL Connect, Order │ │
│ │ News Feed │ │ Event Log │ │ Book, Place Order │ │
│ └─────────────┘ └──────────┬──────────┘ └───────────┬─────────────┘ │
└──────────────────────────────┼──────────────────────────┼────────────────┘
│ │
┌─────────────────────┼──────────────────────────┼─────────────────┐
│ │ Node Server :8080 │ │
│ WebSocket (frames) │ ┌────────────────────┐ │ REST / WebSocket│
│ │ │ │ Oracle Event │ │ │
│ ▼ │ │ Handler │ │ /api/xrpl/* │
│ ┌──────────────────┴──┴────────────────────┴──┴───────────────┐ │
│ │ Stream Watcher :8421 (Python/FastAPI) │ │
│ │ • Frame → Gemini VLM │ │
│ │ • trigger_event() → POST /api/oracle/event │ │
│ │ • LLM suggest markets │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ Gemini API │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ XrplBankManager (Node) ││
│ │ • Escrow (prize vault) ││
│ │ • CREDIT / YES / NO tokens ││
│ │ • Native DEX trading ││
│ └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
│
▼
XRPL Devnet (wss://s.devnet.rippletest.net)
Components
1. Chrome Extension (manifest.json, sidebar/, extension/)
- Sidebar with three tabs: Dashboard, Watch, Trade
- Dashboard: Portfolio (placeholder), News feed
- Watch: Iframe to
extension/stream-watcher/watcher-ui.html— starts session, captures tab, shows event log - Trade: XRPL wallet connect, chart placeholder, order form, order book, suggested markets
2. Stream Watcher (backend/stream_watcher.py)
- FastAPI app on port 8421
- Session lifecycle:
POST /session/start→ creates session, registers with Node - Frame pipeline:
- Extension captures video frame (1 fps) → WebSocket
ws://127.0.0.1:8421/stream/{session_id}- Backend sends frame to Gemini with system prompt +trigger_eventtool - Gemini returns function call if event detected (confidence ≥ 0.8) - Event types:
speedrun_started,entered_nether,got_blaze_rods,found_stronghold,entered_end,ender_dragon_killed,speedrun_ended - Webhook: On boundary events, POSTs to
ORCHESTRATOR_URL/api/oracle/event - Suggestions:
GET /session/suggest-markets/{session_id}— LLM suggests market questions from stream title + recent events
3. Stream Watcher Extension (extension/stream-watcher/)
- watcher.js: Tab capture via
chrome.tabCapture, canvas frame extraction, WebSocket upload, SSE event listener - watcher-ui.js: Auto-starts when on Twitch, displays event cards, stops on
speedrun_ended-type events
4. Node Server (server.js)
- Express + WebSocket on port 8080
- Orchestrator: Receives oracle events, creates/resolves XRPL markets
- XRPL Bank: Bootstrap, create market, issue credit, trade, balances
- Quick-play mode: Virtual balances + in-memory order book for users without XRPL wallet
5. XrplBankManager (xrplBank.js)
- Connects to XRPL Devnet
- Escrow (prize vault): Sponsor locks XRP; winners receive on resolution
- Tokens: CREDIT (trading currency), YES, NO (outcome tokens) per market
- DEX: Native OfferCreate for buy/sell orders
- Settlement:
settleMarket()releases escrow, pays winners proportionally
6. Sidebar Trade Logic (sidebar/sidebar.js)
- Connects to Node WebSocket for market updates
- XRPL mode: Fund wallet, import secret, fetch balances, place orders via REST
- Uses
currentXrplMarketIdfor trades; updates onmarketCreatedbroadcast
Stream → Market Mapping
| Stream Layer | Protocol Layer |
|---|---|
| 1 stream session | 1 oracle session (Gemini watching frames) |
| 1 speedrun attempt | 1 prediction market (run_N) |
speedrun_started |
New market created, liquidity minted |
ender_dragon_killed |
Recorded for YES outcome |
speedrun_ended |
Market resolved: YES if dragon killed, else NO |
| Stream title + events | LLM-suggested market questions |
Resolution logic:
ender_dragon_killedfires → outcome = YESspeedrun_endedwithout prior dragon kill → outcome = NO
XRPL Integration
Primitives Used
- Escrow (
EscrowCreate/EscrowFinish): Prize vault for winners - Issued tokens: CREDIT, YES, NO per market (hex-encoded currency codes)
- Trust lines: Viewers need TrustSet for issuer
- DEX offers:
OfferCreatefor buy/sell YES/NO with CREDIT
API Endpoints
| Method | Path | Description |
|---|---|---|
| POST | /api/xrpl/init |
Initialize xrplBank + issuer |
| POST | /api/xrpl/bootstrap |
Ensure default market exists |
| POST | /api/xrpl/create-vault |
Create escrow prize vault |
| POST | /api/xrpl/create-market |
Create market, mint liquidity |
| POST | /api/xrpl/issue-credit |
Issue CREDIT to viewer |
| POST | /api/xrpl/trade |
Place buy/sell order on DEX |
| GET | /api/xrpl/balances/:address/:marketId |
Get CREDIT, YES, NO |
| GET | /api/xrpl/current-market |
Current tradeable market |
| GET | /api/xrpl/markets |
List markets |
| POST | /api/xrpl/derive-address |
Derive address from secret |
| POST | /testnet/fund |
Fund new Devnet wallet |
Data Flow
Oracle Event Flow
- User opens sidebar on Twitch → Watch tab auto-starts
watcher-ui→POST /session/start→ session created- Stream watcher →
POST /api/session/register(Node) - Frame capture → WebSocket →
process_frame()→ Gemini - Gemini returns
trigger_event(event_type, ...)→ pushed to SSE - On
speedrun_started/ender_dragon_killed/speedrun_ended→POST /api/oracle/event - Node updates run state, creates/resolves markets, broadcasts via WebSocket
Trading Flow (XRPL)
- User clicks "Fund Testnet Wallet" →
POST /testnet/fund→ wallet created - Node bootstraps XRPL, creates default market if needed
POST /api/xrpl/issue-credit→ 1000 CREDIT to user- User places order →
POST /api/xrpl/trade→ OfferCreate on XRPL - Balances fetched via
GET /api/xrpl/balances/:address/:marketId
Setup & Run
Prerequisites
- Node.js 18+
- Python 3.10+
- Chrome (for extension)
GEMINI_API_KEY(Google AI Studio)
1. Extension
- Clone repo, load unpacked in Chrome (
chrome://extensions/→ Load unpacked) - Ensure
manifest.jsonandsidebar/are in the extension folder
2. Stream Watcher (Oracle)
cd backend
cp .env.example .env
# Set GEMINI_API_KEY=your-key
pip install -r requirements.txt
./start.sh # or: uvicorn stream_watcher:app --host 127.0.0.1 --port 8421 --reload
Runs on http://127.0.0.1:8421
3. Node Server (Orchestrator + XRPL)
node server.js
Runs on http://localhost:8080 and ws://localhost:8080
4. Full Stack
- Open Chrome → Twitch (Minecraft speedrun stream)
- Open extension sidebar → Watch tab (auto-starts) → Trade tab (fund wallet, trade)
API Reference
Oracle / Session
| Method | Path | Body | Description |
|---|---|---|---|
| POST | /api/session/register |
{ session_id } |
Register stream watcher session |
| POST | /api/oracle/event |
{ event_type, session_id, description } |
Receive VLM event |
| GET | /api/suggestions |
— | Proxy to stream watcher for market suggestions |
| GET | /api/xrpl/current-market |
— | Current market id, question, status |
WebSocket Messages (Node → Client)
| type | Payload |
|---|---|
init |
balance, markets, orderBook, oracleTime |
marketCreated |
market |
marketResolved |
market |
orderBookUpdate |
book, fills |
balanceUpdate |
balance |
orderAccepted |
order, fills |
Demo Guide
3-Minute Demo Flow
- Open Twitch (Minecraft speedrun stream)
- Open sidebar → Watch tab: show event log (entered_nether, etc.)
- Trade tab → Fund Testnet Wallet → place Buy/Sell
- Explain: Each run = one market;
speedrun_ended→ resolve; YES = dragon killed, NO = death/reset
Features to Highlight
- XRPL: Escrow, issued tokens, native DEX
- Real-time VLM oracle (Gemini vision + function calling)
- Automatic market create/resolve from stream events
- LLM-suggested markets from stream context
- One-click testnet funding
Built With
- fastapi
- gemini-api
- node.js
- polymarket
- python
- twitchapi
- websockets
- xrpl
- xrpl.js
Log in or sign up for Devpost to join the conversation.