Ask questions and chat naturally with Nova.
Displays local weather, including temperature, humidit, and daily high and low temperatures. Users can also search for another city.
Shows a curated playlist with track names, artists, durations, and playback controls.
Lets users create countdown timers and schedule alarms with labels, then view and manage active items.
Combines a spoken daily summary with weather, task status, estimated household energy use, and news headlines.
Allows users to add a task title, due date, priority, and estimated work time to track upcoming submissions.
Lets users save, review, and delete facts, preferences, and routines for Nova to use in later conversations.
Shows daily screen time against a target and displays the current focus status.
Displays appliance controls and estimated household power consumption.

🌟 NOVA AI (Alexa-Next 2.0): Autonomous Multimodal Assistant & Self-Hosted MCP Agent

💡 Inspiration

Since its inception, Amazon Alexa has transformed ambient computing in millions of living rooms and kitchens. However, contemporary voice assistants remain largely trapped in a passive, single-turn paradigm:

Command Reactivity vs. True Agency: An assistant can set a timer or turn on a light, but it cannot autonomously decompose high-level objectives into sequential, verifiable subtasks.
The "Black Box" Trust Dilemma: Users lack visibility into what external tools are invoked, whether actions actually succeeded, or how credentials and sensitive parameters are handled.
Severe Contextual Amnesia: Assistants forget user routines, working styles, and commitments casually stated in conversation between turns.
Lack of Proactive Intelligence: They wait passively to be addressed rather than detecting schedule congestion, approaching deadlines, or physical distress.

When Amazon announced the Build, Ship, Shape: Amazon Developer Hackathon with the Alexa+ Track focusing on agentic experiences and self-hosted Model Context Protocol (MCP) servers, we were deeply inspired. Because official preview developer toolkits are restricted to select enterprise partners, Amazon challenged developers to build self-hosted MCP capability providers and simulate the next-generation Alexa+ multimodal experience.

We set out to build the definitive answer: NOVA AI (Alexa-Next 2.0)—an autonomous, proactive agent pairing an authentic 8-state Animated Alexa Eye with a compliant FastMCP 2025-11-25 Streamable HTTP server, multimodal vision, conversation-driven commitment tracking, and Fire OS/Android companion telemetry.

✨ What It Does

NOVA transforms smart displays (like the Amazon Echo Show and Fire Tablets) into proactive command centers:

🚀 Autonomous Mission Mode: Takes complex, multi-stage goals (e.g., "Help me organize my week, prepare for my midterm assignments, identify conflicts, and create a realistic study schedule") and autonomously formulates an executable plan. It renders interactive plan-review cards on screen where users can rename, reorder, or delete steps before granting approval. Every step features bounded retries and postcondition verification.
🔌 Self-Hosted FastMCP Server (Alexa+ Track): Implements the official Model Context Protocol (v1.30+ / spec 2025-11-25) over Streamable HTTP transport mounted live at /mcp, exposing genuine tools: list_devices, set_device_state, get_energy_summary, and get_saving_tips.
👁️ Authentic 8-State Animated Alexa Eye: A reactive cybernetic canvas visualizer with 8 distinct acoustic and operational states: IDLE (soft cyan breathing), LISTENING (audio waveform), THINKING (indigo pulse), SPEAKING (azure ripple), EXECUTING (atomic spin), SUCCESS (emerald glow), ERROR (crimson strobe), and PAUSED (amber approval orbit).
🗓️ Conversation Commitment & Deadline Extraction: Analyzes natural speech to capture explicit dated promises ("I need to submit my physics assignment tomorrow by 5 PM") without date hallucinations, schedules proactive alerts 1 hour ahead, and scans the past 24 hours of multi-turn chat to unearth forgotten tasks.
🧠 User-Controlled Long-Term Memory (Digital Twin): Persistent SQLite storage for confirmed facts, routines, preferences, and project context. Users retain absolute sovereignty with search, pin, edit, delete, clear, and a master injection toggle.
📄 Multimodal Document & Vision Intelligence: Upload course syllabi, assignment PDFs, images, and text notes. Gemini Vision extracts deliverables, due dates, and grading weights, offering a 1-click + Add to Tasks integration.
⚡ Proactive Conflict Detection & What-If Simulator: Audits schedule density to flag overloaded days. Features an isolated sandbox simulator where users can test shifting deadlines and evaluate trade-offs before committing changes.
🏡 Smart Home IoT & Real Wattage Calculations: Controls household appliances with real mathematical wattage modeling and energy-saving recommendations.
🌅 Executive Daily Briefing, Music & Alarms: Synthesizes local weather (OpenWeatherMap), pending tasks, smart home power load, and live Google News RSS headlines into a spoken morning briefing, alongside audio playback and smart alarms.
🚨 Emergency SOS Life Guardian: Distress siren audio synthesis via Web Audio API, live GPS location capture, reverse geocoding via OpenStreetMap Nominatim, OSRM turn-by-turn routing, and simulated distress dispatch.

🛠️ How We Built It

NOVA was engineered from the ground up for responsiveness, zero build friction, and strict adherence to open standards:

1. Gateway & Protocol Layer

FastAPI (Python 3.10+): Asynchronous REST gateway serving APIs, static UI assets, and WebSocket/HTTP streams.
FastMCP (Streamable HTTP): Implemented using the latest mcp Python SDK complying with the 2025-11-25 MCP specification. We configured a lifespan context manager with transport security allowed hosts (localhost, 127.0.0.1, *) mounted directly at /mcp.

2. Multi-LLM Fallback & Reasoning Architecture

We built a resilient multi-provider LLM chain in core/brain.py: $$\text{Provider Sequence: } \text{Gemini 3.8 Flash} \longrightarrow \text{OpenAI GPT-4o} \longrightarrow \text{Groq Llama 3.3-70B} \longrightarrow \text{Offline Deterministic Engine}$$ If external API quotas are exhausted (HTTP 429) or internet drops, NOVA automatically degrades gracefully to its local deterministic engine without crashing.

3. Mathematics of Ambient Energy & Workload Modeling

Smart-Home Power Draw: Total real-time power load is modeled across active relays: $$P_{\text{total}} = \sum_{i=1}^{N} P_i \cdot \mathbb{I}(\text{device}_i = \text{ON})$$ where $P_i$ denotes the wattage rating of device $i$ (e.g., Living Room Light: 9W, AC: 1500W, TV: 80W) and $\mathbb{I} \in {0, 1}$ represents power state.
Workload Conflict Density: The proactive engine measures daily cognitive congestion: $$\rho(d) = \sum_{t \in \mathcal{T}d} \frac{H(t)}{C{\text{max}}}$$ where $H(t)$ is estimated hours for task $t$ on day $d$, triggering an automated rebalancing recommendation whenever $\rho(d) > 1.0$.

4. Client Dashboard & Fire OS Companion

Echo Show Companion UI: Single-page dashboard built with Vanilla HTML5/CSS3/JavaScript, Tailwind CSS, Lucide icons, and Leaflet Maps, eliminating heavy Node.js build dependencies for judges.
Fire OS / Android Companion: Kotlin background service in mobile_android/NovaForegroundService.kt utilizing FusedLocationProviderClient, UsageStatsManager for screen time, and SmsManager for SOS dispatch.

🧗 Challenges We Faced

FastMCP Transport in Modern Web Environments:
- Challenge: The latest Model Context Protocol specification favors Streamable HTTP transport over standard stdio, requiring proper session lifecycle management and CORS header validation.
- Solution: We integrated mcp.session_manager.run() inside FastAPI's async lifespan context, enabling the same server process to serve REST endpoints, the static frontend, and the MCP JSON-RPC protocol simultaneously.
Preventing AI Hallucinations in Spoken Deadlines:
- Challenge: General-purpose LLMs frequently hallucinate deadlines or invent arbitrary calendar dates when users mention conversational time phrases like "sometime next week".
- Solution: We built a dual-layer regex and natural language date parser (_parse_deadline_datetime) with strict reference-time anchoring, named month resolution, and negative keyword filtering (what if, maybe, might) to ensure only explicit commitments trigger calendar entries.
Multi-Turn Context & Zero Secret Leakage:
- Challenge: Allowing external tools to run safely while guaranteeing zero credential leakage to client browsers.
- Solution: Engineered the Trust Center architecture: 100% of API keys stay strictly server-side in .env. All tool arguments written to the timeline audit log (activity_logs) are sanitized and masked before reaching the browser.

🏅 Accomplishments That We're Proud Of

✅ Official MCP 2025-11-25 Streamable HTTP Compliance: A fully functional, self-hosted FastMCP server ready for agent-to-agent interoperability with Alexa+.
✅ 12/12 Automated Integration Tests Passing: Comprehensive test suite in tests/test_nova.py verifying health diagnostics, mission execution, memory CRUD, what-if simulation, and daily briefings in under 3.5 seconds.
✅ Authentic 8-State Animated Alexa Eye: Fluid cybernetic visualizer with smooth canvas and CSS state transitions.
✅ Non-Destructive What-If Schedule Simulator: True sandbox modeling of schedule density before modifying production databases.
✅ Zero Build Setup Required: Runs out-of-the-box with python run.py, automatically launching the Echo Show companion dashboard.

📚 What We Learned

The Power of Open Protocols: Building with the Model Context Protocol (MCP) demonstrated that future voice assistants will thrive on open standards rather than proprietary, walled-garden skill APIs.
Human-in-the-Loop is Critical for Agency: True autonomous agent architectures must balance capability with user trust. Providing editable plan cards before consequential execution creates a sense of empowerment rather than apprehension.
Proactive Ambient Computing: An assistant that anticipates needs—warning of deadline collisions 24 hours prior and scheduling automatic 1-hour reminders—is exponentially more valuable than one that merely waits for a wake word.

🚀 What's Next for NOVA AI

Hardware Enclosure for Amazon Echo Show Stand: Packaging the web companion into a dedicated kiosk mode for Amazon Fire Tablets and Echo Show devices.
On-Device Small Language Models (SLMs): Integrating quantised on-device models (e.g., Gemma 2B or Llama 3.2 1B via ONNX Runtime / llama.cpp) for 100% offline edge privacy.
Matter & Zigbee IoT Gateway: Extending the FastMCP server tools to directly bridge physical Matter-enabled household smart devices across the home.