HomeOps

Your home always secured

Comment

Inspiration

Smart homes are often fragmented across disconnected apps. A doorbell rings in one app, a camera captures footage in another, and security controls live somewhere else. We wanted to build a single intelligent system that connects these events, understands the context, and takes meaningful action. This inspired HOME OPS — a context-aware home operations agent that observes events, reasons about situations, and acts through Model Context Protocol (MCP) tools, while making its decisions and actions visible on a live TV-style console.

What it does

HOME OPS unifies voice requests, doorbell events, camera detections, and terminal commands through a centralized event bus. An AI agent receives these events along with contextual information such as the time of day and home mode, then decides which actions to take.

For example:

Daytime delivery: Sends an informational alert and speaks a delivery note without changing the home mode.

Friend at the door: Sends an informational alert and delivers a greeting.

Unknown person during the day: Issues a warning while leaving the home mode unchanged.

Unknown person at night or while security is armed: Raises a critical alert, arms all security zones, and speaks a warning.

The agent executes actions exclusively through MCP tools: set_mode, arm_security, disarm_security, speak, and notify. Every tool call generates an event that appears on the live console, providing visibility into the system's activity.

The project also includes a web-based voice and text assistant simulating the Alexa+ experience and a local YOLO-based vision pipeline for camera-frame analysis.

How we built it

We built HOME OPS as an AI-powered smart-home operations agent that connects events, contextual reasoning, and real-world actions through a unified system. We used FastAPI and Server-Sent Events (SSE) to create a central event bus that receives events from the CLI, simulated voice assistant, doorbell simulator, and camera perception pipeline.

The agent is powered by the Strands Agents SDK and uses an LLM through Groq or Gemini to interpret events alongside contextual information such as the time of day and current home mode. We built a separate self-hosted MCP server using FastMCP, exposing five tools: set_mode, arm_security, disarm_security, speak, and notify. The agent acts exclusively through these MCP tools, and every tool call generates an event that appears on our live React-based TV-style console.

For visual perception, we integrated a local YOLO model to analyze AI-generated porch images. We also built a web-based voice and text assistant to simulate the Alexa+ experience. The project is open source under the MIT license, with automated tests that can run without network access or LLM calls.

Challenges we ran into

One of our biggest challenges was managing dependency conflicts between the Strands Agents SDK and FastMCP. Their MCP dependency requirements were incompatible, so we separated the agent and home infrastructure into two Python environments that communicate over HTTP.

We also encountered integration friction while connecting the event bus, agent, MCP tools, camera perception, and live console into a single workflow. Making agent decisions context-aware was another challenge: the same doorbell event needed to produce different responses depending on the visitor, time of day, and security mode.

We ran into limitations with the free-tier LLM APIs, where each agent run requires approximately three to five requests. We also explored the Fire TV/Vega track but had to drop it because the Vega SDK would not run on our development machine. Since we didn't have an Alexa+ device, we built a browser-based simulation instead, allowing us to demonstrate the intended conversational experience without claiming a native device integration.

Accomplishments that we're proud of

We're proud of building a complete, modular smart-home agent that connects multiple event sources to contextual decisions and observable actions. Instead of creating another disconnected smart-home interface, we built a system in which events flow through a shared bus and an AI agent can respond using a controlled set of MCP tools.

One of our key accomplishments is contextual behavior. A delivery during the day results in an informational alert and spoken note, while an unknown visitor at night triggers a critical alert, security arming, and a spoken warning. This demonstrates how context can change the system's response to the same type of event.

We're also proud of the live TV-style console, which makes events and agent actions visible; the integration of local YOLO-based visual perception; the simulated voice assistant; and the separation of the agent from the MCP server. Finally, we made the project open source under the MIT license and included tests that don't require an external LLM or network access.

What we learned

We learned that building a useful AI agent involves much more than connecting an LLM to a set of tools. Reliable event handling, contextual state, tool boundaries, dependency management, and observability are equally important.

We gained practical experience with the Strands Agents SDK, MCP client-server architecture, FastMCP, streaming events with FastAPI and SSE, and local computer vision using YOLO. We also learned the value of isolating incompatible dependencies into separate environments and using HTTP interfaces to connect independently managed components.

Most importantly, we learned that smart-home automation needs to be context-aware and safety-conscious. An agent should not simply react to an event; it should consider the situation, choose an appropriate action, and make that action observable. Our experience also reinforced the importance of distinguishing a simulated integration from a fully tested hardware deployment.

What's next for HomeOps

Our next step is to move HOME OPS from a simulated prototype toward a more reliable, real-world smart-home assistant.

Real smart-home integrations: Connect to actual cameras, doorbells, speakers, and home automation platforms.

Native voice assistant support: Explore a real Alexa+ integration and compatible device workflows beyond our browser simulation.

Safer automation: Introduce deterministic security policies, stronger authorization, explicit confirmation for sensitive actions, and safeguards against incorrect or repeated tool calls.

Better perception and context: Incorporate live camera streams, more reliable visitor classification, and richer home-state information.

Improved observability: Expand the console with event timelines, decision summaries, tool execution results, and clear error reporting.

Greater reliability: Add event persistence, recovery after service restarts, idempotent actions, and broader scenario-based testing.

Simpler deployment: Provide an easier setup process, improved dependency management, and a streamlined way to launch the entire system.

Our long-term vision is for HOME OPS to become an open-source, context-aware home operations agent that brings disconnected smart-home devices and services together, explains its decisions, and takes appropriate actions through a secure, extensible tool interface.

Built With

Share this project:

Updates

Submission history