🛡️ Amazon OmniHub: Cross-Device Ambient Intelligence Suite

Built for "Build, Ship, Shape: Amazon Developer Hackathon 2026"

🌟 Executive Summary

Amazon OmniHub is a unified ambient intelligence system connecting Alexa+ Agent Skills (Model Context Protocol), Ring Smart Sentinel Cameras, and Fire TV 10-foot Living Room UI, driven by AWS Bedrock multimodal generative reasoning.

Rather than treating smart home devices as isolated siloes, OmniHub transforms them into a collaborative ambient mesh:

Ring Doorbell Pro detects front door activity (courier delivery, neighbor greeting, or suspicious loitering).
AWS Bedrock Vision Models (amazon.nova-pro-v1 / anthropic.claude-3-5-sonnet) autonomously triage the video frame in under 200ms.
Alexa+ Agent Engine calls Model Context Protocol (MCP) tools to coordinate cross-device actions.
Fire TV in the living room instantaneously adapts: displaying Picture-in-Picture (PiP) security stream overlays, adjusting ambient lighting scenes (Cinema Mode, High Alert, Relax), and accepting 10-foot D-Pad remote control commands.

🏆 Hackathon Tracks Qualified

Track / Mini Challenge	How OmniHub Qualifies
Alexa+ Track	Full implementation of an Alexa+ Agent Skill leveraging the Model Context Protocol (MCP) with 7 registered tools, multi-turn conversational reasoning, speech synthesis, and chain-of-thought tool calling.
Fire TV Track	Native 10-foot Living Room Experience designed for Fire OS / Vega OS with remote control D-pad navigation, ambient backdrop adaptation, Picture-in-Picture (PiP) doorbell feeds, and IoT control tiles.
Ring Track	Real-time front-door stream triage, automated 2-way outdoor intercom broadcasts, deterrent siren triggers, and intelligent parcel/visitor detection.
AWS Builder Challenge	Full cloud orchestration with AWS Bedrock generative AI inference, multi-modal vision prompt engineering, latency profiling, and token optimization.
Open Source Challenge	Complete, modular, open-source codebase with standalone MCP server (`mcp-server.js`) compliant with standard JSON-RPC 2.0 specifications.

📐 Architecture & Data Flow

flowchart TD
    subgraph FrontDoor["🚪 Front Door Ecosystem (Ring Sentinel)"]
        Cam["Ring Doorbell Pro (1080p HDR)"]
        Sensor["Motion & PIR Sensor"]
        Speaker["Two-Way Outdoor Intercom"]
        Lock["Ring Smart Deadbolt"]
        Light["1800-Lumen Porch Floodlight"]
    end

    subgraph AWSCloud["☁️ AWS Cloud & Bedrock"]
        Bedrock["AWS Bedrock Runtime\n(amazon.nova-pro-v1 / Claude 3.5)"]
        EventStream["Amazon EventBridge / SQS Stream"]
        Telemetry["CloudWatch & Latency Metrics"]
    end

    subgraph AlexaAgent["🎙️ Alexa+ Agent (MCP Server)"]
        Voice["Alexa+ Multi-Turn Speech Engine"]
        MCP["Model Context Protocol Server\n(JSON-RPC 2.0 / 7 Tools)"]
        Reasoning["Chain-of-Thought Decision Pipeline"]
    end

    subgraph LivingRoom["📺 Living Room 10-ft (Fire TV Omni)"]
        FireOS["Fire OS / Vega OS 10ft Dashboard"]
        PiP["Picture-in-Picture Security Window"]
        DPad["CEC / Bluetooth Remote Navigation"]
        Scenes["Ambient Atmosphere Backlight"]
    end

    Sensor -->|Motion Trigger| Cam
    Cam -->|Snapshot Payload| Bedrock
    Bedrock -->|Threat & Intent Classification| EventStream
    EventStream --> MCP
    MCP --> Reasoning
    Reasoning --> Voice
    Reasoning -->|firetv_show_pip| PiP
    Reasoning -->|firetv_set_scene| Scenes
    Reasoning -->|smart_home_set_lock| Lock
    Reasoning -->|smart_home_set_light| Light
    Voice -->|ring_trigger_quick_reply| Speaker
    DPad --> FireOS

🛠️ Model Context Protocol (MCP) Tools

OmniHub exposes 7 registered MCP tools for the Alexa+ Agent:

ring_get_doorbell_status: Retrieve battery level, Wi-Fi RSSI, sensor telemetry, and last motion timestamp.
ring_triage_visitor: Multimodal vision inference evaluating visitor identity, package presence, and threat tier (SAFE, INFO, FRIENDLY, HIGH_RISK).
ring_trigger_quick_reply: Broadcast natural voice quick-replies through the Ring outdoor speaker.
firetv_show_pip: Dynamically pop up or expand a live video Picture-in-Picture window on Fire TV.
firetv_set_scene: Transition living room ambient scenes (normal, cinema, security_alert, relax).
smart_home_set_lock: Remotely verify and toggle front door deadbolt lock state.
smart_home_set_light: Adjust porch floodlight illumination mode and brightness.

🚀 Quickstart & Running Locally

Prerequisites

Node.js v18+ (tested on Node.js v24)
npm v9+

1. Install Dependencies

npm install

2. Start the Interactive Dev Application

npm run dev

Open http://localhost:5173/ in your browser.

3. Run the Standalone MCP Server (Stdio JSON-RPC 2.0)

npm run mcp

4. Build for Production

npm run build

🎮 How to Test & Demo

Fire TV 10-ft Experience:
- Use your keyboard Arrow keys (▲ ▼ ◀ ▶) and Enter to navigate focusable buttons just like a living room remote control.
- Click D-Pad Remote in the top navigation to toggle an on-screen Fire TV virtual remote.
- Click the Scene buttons (Cinema Mode, High Alert, Relax) to observe dynamic living room ambient backlight transitions.
Ring Sentinel AI Vision Triage:
- Switch to the Ring Sentinel tab.
- Click scenario presets:
  - 📦 Amazon Delivery: Identifies parcel drop-off, locks door, sets low-risk status.
  - 👋 Neighbor (Sarah): Matches friendly visitor, offers two-way intercom greeting.
  - 🚨 Night Intrusion: Detects loitering prowler with infrared vision, triggers floodlight, flashes Fire TV alert.
  - 🌿 Clear Porch: Nominal baseline.
- Click Ring Doorbell to hear audio chimes and trigger automatic Alexa+ voice triage.
Alexa+ Conversational Agent:
- Switch to the Alexa+ Agent tab.
- Speak via microphone or click suggested voice chips:
  - "Alexa, who is at the front door?"
  - "Alexa, activate Movie Night mode on Fire TV"
  - "Alexa, is the front door locked?"
- Watch the animated audio waveform, review chain-of-thought reasoning steps, and inspect the active MCP tool call JSON payload.
Model Context Protocol & AWS Telemetry:
- Switch to the MCP & AWS Telemetry tab.
- Directly click Test Invoke Tool on any of the 7 tools to trigger real-time JSON-RPC 2.0 messages.
- Inspect live latency benchmarks (~180ms), token count, and protocol logs.

💡 Constructive Product Feedback for Amazon Developer Teams

(Submitted as part of the official Devpost hackathon requirements)

1. Fire TV & Vega OS Developer Experience

Unified Remote Input Emulation: Web apps running on Fire OS often require manual mapping of keydown keycodes for the D-pad and Back buttons. Providing a standardized @amazon/firetv-navigation npm package with focus management and spatial navigation out of the box would significantly lower the barrier for web developers.
Picture-in-Picture (PiP) Web API: Standardizing an official Fire TV Web API for requesting system-level PiP overlays (like Ring doorbell alerts during video playback) will unlock seamless cross-device ambient notifications for third-party developers.

2. Alexa+ & Model Context Protocol (MCP) Integration

First-Class MCP Tool Registry in Alexa Developer Console: Alexa+ adopting Model Context Protocol is an enormous leap forward. It would be revolutionary if developers could register MCP servers via standard URL/SSE endpoints directly inside the Alexa Developer Console without needing redundant intent schema JSON mappings.
Proactive Ambient Event Streaming: Adding a low-latency push event channel from external IoT devices (like Ring motion alerts) to wake up an Alexa+ ambient session without requiring an explicit wake-word prompt.

3. Ring Developer APIs

Two-Way WebRTC Audio Stream Hooks: Enabling programmatic synthetic text-to-speech injection into the Ring outdoor speaker via REST/WebSocket would empower AI agents to conduct intelligent visitor triage conversations autonomously.