Predictit — Look at the Game. See the Probability.

Inspiration

Prediction markets are designed to turn information into probabilities, but using them during a live event still feels surprisingly manual. A viewer has to stop watching, open another device, search for the correct event, find the right contract, interpret the current price, and mentally connect it back to what just happened on screen.

That interaction is especially awkward during sports, where the state of the game can change in seconds.

We asked a simple question:

What if the broadcast itself could become the search query?

Predictit transforms Meta Ray-Ban glasses into a hands-free visual interface for prediction markets. Instead of searching through menus, the user looks at a game. The system recognizes the sport, selects the corresponding event, and presents a concise probability analysis alongside historical market context.

The result is an experience where accessing prediction intelligence feels as natural as watching the event itself.

What it does

Predictit connects a Meta glasses camera feed to a desktop prediction interface.

When the user looks at a television or laptop showing a supported sport, Predictit:

Streams the glasses camera directly to the desktop through WebRTC.
Captures a single, lightweight analysis frame after the user presses Analyze.
Uses Gemini 3.5 Flash to identify the content. We than figure out the exact match and find a recording/stream.
Maps the detected sport to one of four historical showcase broadcasts.
Replaces the raw camera preview with the matching high-quality stream.
Displays a backtested probability, an illustrative historical market price, the model-market difference, supporting evidence, and relevant research.
Continues monitoring the glasses feed in the background.
Automatically switches broadcasts when the user begins watching another supported sport.

There is no sport selector, market search, or chatbot. The camera is the interface.

The showcase events

Our hackathon experience supports four iconic events currently.

Each event has a deterministic historical intelligence pack containing probability checkpoints, illustrative replay-market prices, evidence, strategy context, key factors, alternate markets, and a complete probability timeline.

Because these events have already happened, we can evaluate and refine the entire experience against known game states instead of pretending that an untested model is producing perfectly calibrated live probabilities.

However, we deliberately avoid leaking the final result into earlier predictions. Each probability represents a point-in-time estimate based on the state visible at that moment. In-between states can be interpolated from adjacent archived checkpoints, while completed events resolve to their final outcome.

How we built it

Predictit is composed of five major systems.

1. Meta glasses capture

Our SwiftUI companion app integrates with the Meta Wearables Device Access Toolkit to access the Ray-Ban camera stream.

The app handles:

Meta AI registration
Camera permissions
Device-session lifecycle
Raw and HEVC frame decoding
Reconnection behavior
Background keep-alive
WebRTC publishing
Restrictive university-network conditions

The glasses act only as the camera provider. Hardware-specific code remains isolated from the prediction and playback systems.

2. Direct WebRTC video

The continuous camera feed travels directly between the mobile device and desktop viewer using WebRTC.

Cloudflare is used for secure application access and signaling—not as a video relay. SDP and ICE metadata pass through the server, while the continuous media uses the WebRTC connection.

TURN support provides a reliable fallback on university networks with client isolation or restrictive NAT behavior.

This separation gives us:

Lower latency
Better video quality
Reduced server bandwidth
A hardware-independent backend
A consistent path for both phone and glasses cameras

3. Lightweight visual detection

We initially experimented with detailed scoreboard extraction, team recognition, temporal frame windows, and multiple vision-model calls. For the final showcase, we realized that this was unnecessary complexity.

Predictit therefore sends one compressed 960-pixel JPEG to Gemini 3.5 Flash through a Gemini-compatible RightCodes endpoint.

The first usable frame is analyzed immediately. Later checks are rate-limited, and each request contains exactly one image. The model only returns a sport classification and confidence score.

This narrow task is faster, cheaper, and more reliable than asking a multimodal model to reconstruct an entire scoreboard when the immediate product decision is simply which broadcast to select.

4. Server-owned playback intelligence

A central PlaybackDirector owns all stream changes.

At 70% classification confidence or higher, the director can load the matching local broadcast. Repeated detections of the same sport never restart or reseek the video. Only a newly detected event creates a new playback revision.

Low-confidence or unclear observations preserve the existing theater state rather than creating a disruptive false switch.

Each local broadcast is pinned through a manifest containing:

Filename
File size
Duration
SHA-256 hash
Media route
Calibrated checkpoint offsets

This makes the demo deterministic and protects it from silently playing the wrong edit or seeking to an invalid timestamp.

5. Prediction intelligence and presentation

Each supported event has a historical intelligence pack containing:

Model probabilities
Illustrative market-implied probabilities
Model-market gaps
Key factors
Evidence
Player and team context
Strategy research
Historical news context
Alternate prediction questions
Probability-change explanations
Upcoming probability triggers

The interface presents this information through a light-mode liquid-glass design inspired by modern spatial-computing interfaces.

A large theater view keeps the event central, while translucent intelligence tiles show the prediction, research, system state, and supporting context.

WebGPU-powered YOLO tracking runs locally in the browser to draw continuous player detections without consuming vision-model quota. The tracking layer is visually compelling but intentionally separate from event selection and probability generation.

What makes Predictit different

Most prediction-market products begin with a list of contracts.

Predictit begins with the physical world.

The user does not need to know the market title, league identifier, team abbreviation, or platform taxonomy. They simply look at the event they care about.

That interaction model can extend far beyond sports:

Elections shown on television
Award ceremonies
Esports tournaments
Financial broadcasts
Product announcements
Court decisions
Live cultural events

The long-term opportunity is not just “prediction markets on glasses.” It is a general visual layer that connects real-world events to probabilistic information.

Challenges we faced

Getting Meta glasses video to the analysis pipeline

Our first Meta implementation successfully displayed video over WebRTC but did not submit frames for visual analysis. The continuous video and HTTP analysis paths had become disconnected.

We fixed this by having the desktop viewer sample the incoming WebRTC track directly. This avoided requiring another native-app build and made detection work consistently across both the phone and Meta glasses providers.

University-network restrictions

Local-device discovery and direct peer connections are unreliable on university Wi-Fi. Devices may be isolated even while connected to the same network.

We separated secure application access from media transport and added TURN-backed WebRTC connectivity, allowing the experience to survive restrictive NAT and client-isolation policies.

Model latency and quota pressure

Our early design packed several frames into every vision request and asked the model to extract teams, scores, clocks, competitions, participants, and game state.

This was slow and wasteful.

We reduced the problem to one frame, five possible labels, and one confidence value. That simplification dramatically reduced payload size and removed unnecessary model work.

Video quality versus inference speed

Player detection benefits from high-resolution video, while remote vision classification benefits from small payloads.

We separated those concerns:

High-quality WebRTC video for the viewer and local WebGPU tracking
A compressed 960-pixel snapshot for remote classification

Keeping a deterministic demo honest

Historical games give us complete information, but simply assigning 100% to the eventual winner at every earlier moment would be misleading.

We built point-in-time historical probability timelines instead. Each estimate is tied to the state of the event at that moment, and all replay prices and research are visibly labeled.

Accomplishments we are proud of

Built a working Meta Ray-Ban-to-browser video pipeline
Achieved cross-device WebRTC connectivity on a restrictive university network
Automatically classify four distinct sport categories without user selection
Switch between four high-quality local broadcasts from glasses input
Reduced detection to one image per model request
Built a server-owned, revision-safe playback state machine
Created deterministic probability timelines for four iconic events
Added clearly labeled model-versus-market comparisons
Implemented local WebGPU YOLO player tracking
Designed a distinctive light-mode liquid-glass interface
Built phone, desktop, iOS, data, landing, and pitch surfaces
Created a quota-free rehearsal mode for reliable judging
Maintained a comprehensive automated test suite across the vision, API, playback, market, and probability layers

What we learned

The most important lesson was that stronger products do not always require larger prompts or more complicated models.

Our breakthrough came from aggressively narrowing the vision model’s responsibility. Once we recognized that the product decision was “which of four broadcasts should be selected,” the correct model output became a five-way classification—not a complete semantic reconstruction of the game.

We also learned that wearable applications require careful separation between media, control, and intelligence:

WebRTC is excellent for continuous media.
HTTP is better for sparse analysis snapshots.
The backend should own irreversible state transitions.
The client should remain replaceable.
Hardware integration should never leak into the prediction engine.

Finally, we learned that credibility matters as much as visual polish. Historical probabilities, mock market prices, and cached research can create an excellent showcase—but only when their provenance is presented clearly.

What is next

The current showcase intentionally maps one sport to one historical event. The next stage would replace that deterministic mapping with a two-step retrieval system: