Mesh (or Amazon Continuity Hub)

The Core Problem & VisionThe Problem (Device Amnesia): Users own Echo speakers, Fire TVs, and laptops running browsers like Firefox, all logged into the same account. However, their developer APIs are completely siloed. Your Echo has zero visibility into the code or article open on your laptop, and your browser has no idea where you paused a show on your Fire TV. Moving tasks across screens requires manual casting, clunky menus, or emailing links to yourself.The Solution (Ambient Continuity): EchoMesh acts as a central cloud nervous system. Instead of building fragile point-to-point connections between individual apps, every device connects to a unified WebSocket and cloud state hub. Whenever an action happens on one device, every sibling device receives the updated context in under 30 milliseconds.2. Four Flagship Features & Workflows1. Voice-to-Screen Handoff (Firefox $\rightarrow$ Alexa $\rightarrow$ Fire TV)How it works: While watching a tutorial or reading an article in Firefox, the lightweight Firefox Extension streams the active URL, title, and playback timestamp to the EchoMesh backend over WebSockets.User Action: You say, "Alexa, throw my browser tab to the TV."Result: Alexa triggers the backend webhook, pulls the live Firefox state from DynamoDB, and pushes an instant OPEN_MEDIA WebSocket command to the Fire TV client, which immediately launches the content at the exact timestamp.2. Reverse Ambient Media Sync (Fire TV $\rightarrow$ Firefox)How it works: When you pause a video on Fire TV to walk over to your desk, the Fire TV client sends a STATE_UPDATE event (status: "paused", timestamp_sec: 1340) to the backend.Result: EchoMesh instantly broadcasts that state delta to your Firefox browser, which displays a non-intrusive "Continue Watching from 22:20" banner on your laptop.3. Cross-Device AI Reasoning (Powered by Amazon Bedrock)How it works: Because EchoMesh aggregates the real-time JSON state of all your devices in one place, it feeds that unified context directly into Amazon Bedrock (Claude / Amazon Nova) when you ask a question.User Action: Highlight a block of documentation or code in Firefox and say, "Alexa, explain what I have highlighted on my laptop right now."Result: Alexa reads the selected_text field from your Firefox state via Bedrock and speaks back a clear, context-aware explanation.4. Universal Shared Clipboard & Smart Room TriggersHow it works: Copy a link, promo code, or snippet on your laptop, and it immediately populates the shared_clipboard field accessible by Fire TV (for one-click search/login) and Alexa. Pausing or playing media on Fire TV can simultaneously trigger simulated smart home lighting changes.3. System Architecture & Tech StackArchitectural LayerTechnology UsedRole in EchoMeshCore Backend & HubPython 3.11+ & FastAPIHigh-concurrency async server handling both REST webhooks and persistent WebSocket device rooms.Real-Time TransportNative WebSockets (wss://)Maintains low-latency, bi-directional connections with Firefox, Fire TV, and IoT nodes for sub-30ms state pushes.Persistent Cloud StateAmazon DynamoDB (boto3)Stores the single source-of-truth UnifiedContext JSON document per user (AmazonContinuityState table).Voice InterfaceAlexa Skills Kit (ASK)Translates spoken commands (ThrowBrowserToTVIntent, AskContextIntent) into JSON webhooks sent to FastAPI.Cross-Device AIAmazon Bedrock (converse API)Processes the live multi-device JSON state to answer cross-screen user queries via voice or web.Client NodesFirefox WebExtension (Manifest V3) & Fire TV Web/App ClientLightweight edge clients that report local screen state and execute incoming remote commands.Demo ResilienceDual-Mode Storage Adapter (USE_AWS)Switches seamlessly between local in-memory state (for instant offline testing) and live AWS DynamoDB + Bedrock.4. End-to-End Data Flow (Step-by-Step)Device Registration: When the user opens Firefox or turns on Fire TV, each client connects to WS /ws/{user_id}/{device_id}. The backend marks the device "online" in DynamoDB and sends back the current global state.State Ingestion: As the user switches tabs, highlights text, or scrubs through a video, the client sends a lightweight "STATE_UPDATE" JSON frame over the WebSocket.Persistence & Diff Broadcast: The FastAPI hub merges the change into the user's Unified Context Document in Amazon DynamoDB and immediately broadcasts a "STATE_SYNC" event to all other connected devices in that user's room.Command Execution: When an Alexa voice intent or manual handoff button is triggered, the backend sends a targeted "COMMAND_TRIGGER" payload over the specific target device's WebSocket, causing the target screen (e.g., Fire TV) to react immediately without polling.