OmniPulse TV

Inspiration

Watching live sports in the living room is still a mostly passive, one-size-fits-all experience. Hardcore tactics fans want formation data and play breakdowns, casual family viewers want a clean lounge experience, latecomers need a quick "Catch Me Up" summary, and active viewers want to engage physically. Meanwhile, most living-room web apps treat TV interfaces like desktop browsers rather than designing for 10-foot UI legibility and directional remote (D-pad) navigation.

We built OmniPulse TV to reimagine the living room screen as a remote-first sports intelligence companion for Fire TV—combining tactical exploration, household-aware personalization, resilient AI explanations, and optional local pose coaching into one cohesive interface.

What it does

Remote-First Watch Hub & Household Modes: Tailors the living room experience with persistent household/profile preference memory across three distinct modes: Analyst Mode, Family Lounge, and Catch Me Up.
Interactive Sports X-Ray Tactical Field: Lets viewers navigate player nodes directly using directional arrow/D-pad keys or pointer controls to inspect player positioning, speeds, and tactical spacing.
Resilient AI Play & Player Breakdowns: Supports voice commands (with a built-in typed command fallback) like "Break down this play" or "Who is player 07?". Explanations run through a strict provider boundary—using deterministic rules by default and optionally scaling to Amazon Bedrock grounded in retrieved event context, with automatic fallback to deterministic rules if cloud inference is unconfigured or unreachable.
Consent-Based Active Coach: An optional local pose-coaching experience powered by browser-side MediaPipe Pose Landmarker that processes camera input locally only after explicit user consent.
Transparent Data Provenance: Clearly labels sample vs. provider data across the UI and API responses so deterministic demo fixtures are never misrepresented as official live match feeds.

How we built it

TV Frontend (React + TypeScript + Vite): Engineered specifically for 10-foot TV ergonomics using a high-contrast visual system (Obsidian Black #0B0F14 canvas, Cloud White #F5F7FA typography, Electric Lime #C6F36B focus rings, and Copper #B8734E accents). Includes custom remote-first focus navigation, a Web Speech API adapter with typed fallback, client-side MediaPipe Pose integration, and a dedicated /landing route.
Backend API (FastAPI + SQLAlchemy + Alembic): Provides structured endpoints for household preferences, viewing sessions, tactical fixtures, analysis history, consent-based coach sessions, user feedback, and full household-data deletion. Uses SQLite for zero-friction local startup and PostgreSQL-compatible models with Alembic migrations for production deployment.
Authentication & AWS Architecture: Supports a frictionless local demo token flow (HS256) alongside production-ready Amazon Cognito authentication (RS256 access-token verification checking issuer, token_use=access, and client_id with PKCE onboarding that provisions one household profile per Cognito subject).
AI Provider Boundary (Amazon Bedrock + Deterministic Engine): Designed a clean service abstraction where AI_PROVIDER=bedrock leverages Amazon Bedrock for contextual tactical synthesis while guaranteeing 100% uptime and verifiable output via deterministic rule fallback.
Quality & DevOps: Containerized with multi-service Docker Compose, backed by pytest API/integration suites, Vitest frontend unit tests, and automated GitHub Actions CI pipelines.

Challenges we ran into

Designing for Directional Remote Navigation: Translating an interactive 2D tactical pitch into a predictable D-pad focus graph required careful state management so users never get trapped or lose visual focus on which player node or action CTA is active.
Hardware & Browser API Variance on TV Devices: Remote microphone hardware does not uniformly expose standard browser Web Speech APIs to web apps, and camera/GPU capabilities vary widely across displays. We solved this by building graceful capability detection: automatic typed command fallbacks for voice, explicit hardware/consent gates for Active Coach, and clear status indicators.
Preventing AI Hallucinations in Sports Analytics: Generative models can invent statistics if left unconstrained. We enforced strict context grounding from the fixture database, preserved explicit data-source labels (sample vs. provider) in every payload, and ensured seamless fallback to deterministic analysis.

Accomplishments that we're proud of

Production-Oriented Engineering Hygiene: Rather than building a fragile hackathon mockup, we built a reproducible, tested foundation complete with Cognito RS256 verification, Alembic migrations, Docker orchestration, household data deletion endpoints, and automated CI checks.
10-Foot UI & Visual Accessibility: The combination of high-contrast Electric Lime focus states against an Obsidian Black backdrop makes navigating complex tactical overlays effortless from across the room.
Honest Verification & Provenance: Maintaining strict architectural separation between deterministic sample fixtures and live provider adapters ensures the platform is immediately runnable by judges while remaining genuinely architected for production hardening.

What we learned

Building for TV web runtimes requires treating remote focus, hardware availability (mic/camera/GPU), and fallback paths as first-class architectural requirements rather than afterthoughts.
Pairing deterministic domain rules with cloud LLMs via Amazon Bedrock gives users the best of both worlds: rich natural-language tactical synthesis when connected, and instant, zero-cost reliability when offline or throttled.

What's next for OmniPulse TV

Licensed Sports Data Integration: Building out the live provider adapter to ingest real-time match tracking feeds with compliant league attribution.
Native Fire TV / Vega Platform Verification: Testing and binding native Fire TV platform voice intents and directional remote hardware events on physical target devices and official simulators.
Expanded Multi-Sport Tactical Overlays: Extending the Sports X-Ray pitch and Active Coach drills to support basketball, cricket, and American football formations.