PlayMind Labs Scoring AI

• 

  output from the generation of 2 operators and 1 bettor

• 

  what the scoring methods are that can be edited by ai over time

Inspiration

Tilt isn't universal. A bet-size spike that's alarming for a casual, low-stakes player is Tuesday for a VIP high-roller — but most behavioral risk tooling in sports betting applies one static threshold to everyone. We wanted to build the infrastructure layer that treats calibration as a first-class, human-governed decision: an agent proposes how this specific operator's players should be scored, a human approves it, and then real-time behavior gets evaluated against that operator's actual approved configuration — not a one-size-fits-all rule.

The "aha" moment was realizing this needed to be provably fair to demo: if you want to show that calibration is what matters, you have to hold the underlying behavior constant and vary only the configuration. That's the whole pitch — same signals, different verdicts, because tilt isn't universal.

What it does

Two synthetic sportsbook operators — a conservative, high-frequency, low-stakes book and a high-tolerance VIP book — each have their own historical session data. A calibration agent (Nebius Token Factory, Qwen3-235B) analyzes each operator's data and proposes:

1. Per-signal intervention thresholds and actions, written as real rows into a production-shaped intervention_rules table
2. Per-operator signal weights — how much each of the 7 behavioral signals should count toward that operator's composite risk score

A human reviews both (thresholds, actions, weights, and the agent's rationale for each) and approves them individually. Only approved configs go live. Once both operators are approved, one scripted player session — designed to feel like a real, erratic bettor rather than a scripted demo — streams live through both operators' approved configurations simultaneously. Same underlying behavior, genuinely different verdicts, in real time. A lightweight copilot (Qwen3-30B, fast loop) also watches for tier escalations and suggests a protective next action for a human analyst.

How we built it

• Frontend: React + Vite + Tailwind, polling-based live comparison view
• Backend: Fastify + TypeScript, a 7-signal behavioral severity engine (bet-size escalation, loss chasing, frequency spikes, session extension, deposit patterns, odds shopping, multi-account proxy)
• Calibration agent: Nebius Token Factory, Qwen3-235B-A22B-Instruct-2507, grounded with Tavily search for retention/engagement industry context, strict JSON-schema prompting with defensive parsing
• Persistence: a real Supabase intervention_rules table (not a mock) — we reverse-engineered the actual production schema and column conventions, and wrote against them directly
• Live copilot: Qwen3-30B-A3B-Instruct-2507 for a fast, cheap suggestion loop
• Severity computation is intentionally operator-agnostic — the same raw signals get computed once per tick, then evaluated twice: once against each operator's own approved thresholds and weights. That symmetry is what makes the divergence meaningful instead of arbitrary.

Challenges we ran into

• Emergence Craft turned out to be the wrong tool for orchestration. We loaded every tool it exposed and found it's a SQL/data-catalog assistant, not a workflow engine — so we committed to plain code early and didn't revisit it.
• Mid-build schema pivot. We discovered our calibration agent had been writing to a generic JSON config instead of the real intervention_rules table. Redirecting meant reverse-engineering the real column types, the actual action_procedure catalog already in use, and safely creating new demo-oching the real production operator already in that table.
• A scoring bug that looked like "the model is broken." Our rolling-window recency logic used a hard cutoff, so a player's risk score would snap instantly from maxed-out to fully clear the moment a chase evenaged out of the window — instead of fading. Traced it down to the exact tick and replacedl decay.
• Caught our own copilot suggesting the wrong thing. After a tier-3 cooldown escalation, our first prompt had the copilot suggest offering the player a bonus to keep them engaged — which actively works againsthe product's actual purpose. We rewrote the prompt to enforce protective-only suggestionpending limits) and explicitly forbid anything that increases betting.
• State durability. In-memory proposal rationale didn't survive a server restart even though the real Supabase approval state did, producing a confusing "looks unconfigured" UI. Fixed by falling back to live Supabase data whenever the richer in-memory context is gone.

What we learned

Calibration is a two-layer problem, not one: thresholds decide when a signal fires, but wuld matter for this population — and conflating the two hides real behavioral differencesbetween operators. We also learned that smooth, clean synthetic data hides scoring bugs that only show up under erratic, realistic input — the moment we made our test bettor's behavior genuinely volatile, several silent issues (hard-cutoff snapping, dead-weight signals) became obvious. And moson needs to be checked against product intent, not just schema validity — the copilot'sbonus-suggestion bug was syntactically perfect JSON and semantically wrong.

Built With

• fastify
• nebius
• node.js
• postgresql
• qwen
• react
• supabase
• tailwindcss
• tavily-api
• typescript
• vite