-
-
Architecture Diagram
-
Active Streamable HTTP MCP Server
Inspiration
Most smart assistants today still feel trapped in a single-turn box. They are great at telling you the weather or switching off a smart bulb, but the moment you ask them to handle a real, multi-step chore across different apps and devices, the illusion breaks. You get hit with "I don't understand" or a generic web search link. When the Model Context Protocol (MCP) and Agent Skills became available for modern assistant ecosystems, we realized we finally had the missing bridge between conversational intelligence and real execution. We wanted to build something that moves beyond a passive voice chatbot—an assistant that can actually observe context, break down messy, open-ended goals, and safely run tools across an entire digital setup without requiring constant hand-holding.
What it does
OmniSync AI is an autonomous agent framework and self-hosted MCP hub designed to handle cross-device, multi-step operations seamlessly: Goal-Oriented Task Planning: Instead of running pre-scripted macros, the agent analyzes natural language requests, decomposes them into structured sub-tasks, and selects the right tools for the job. Open Standard MCP Hub: Operates over compliant Streamable HTTP transports, allowing live, bi-directional communication between the core reasoning engine and connected external endpoints. Contextual Device & Cloud Orchestration: Connects disparate services into a unified execution loop—whether that means reconciling calendar invites, pulling cloud infrastructure health telemetry, routing urgent alerts, or tracking project milestones. Self-Reflective Validation: Before taking actions that change state or modify data, the agent checks parameters against deterministic schemas. If a tool call fails or returns an unexpected error, the agent catches it, adjusts its strategy, and tries an alternative path autonomously.
How we built it
Backend & Server Engine: Built using Python and FastAPI, implementing the official MCP SDK to expose dynamic resources, prompts, and tools over persistent Streamable HTTP connections. Reasoning Layer: Integrated large language models via Amazon Bedrock, utilizing structured function calling and strict Pydantic schemas to prevent hallucinated arguments and ensure reliable parameter typing. Agent Skills Manifests: Configured native skill definitions that establish conversational triggers, intent mapping, and granular user authorization scopes. Interactive Test Harness: Developed a companion web visualizer in React and Tailwind CSS that streams active sessions, showing intermediate agent reasoning steps, outgoing MCP payloads, and returned tool data in real time. Packaging & Infrastructure: Containerized the entire pipeline with Docker, deploying resilient cloud endpoints with automated health check monitors to maintain constant stream uptime.
Challenges we ran into
Managing Streamable HTTP State: Transitioning from stateless HTTP request-response cycles to long-lived, streaming transport sessions required building careful connection lifecycle management and error recovery to handle unexpected network drops gracefully. Taming End-to-End Latency: Chaining multiple tool calls together can quickly make an assistant feel slow. We had to optimize prompt sizes, stream partial model tokens immediately, and execute independent data-fetching tools in parallel rather than sequentially. Schema Adherence Under Edge Cases: When users gave ambiguous or incomplete instructions, models occasionally attempted to invent missing tool parameters. We solved this by implementing strict input validation middleware that prompts the model to self-correct before any action hits the live tool.
Accomplishments that we're proud of
Designed and deployed a fully compliant, production-ready MCP server that cleanly implements the latest streaming specifications. Engineered an autonomous error-recovery loop where the agent detects external API errors and self-corrects its execution path without crashing. Built an intuitive live visualizer that demystifies agent reasoning, allowing anyone to watch the agent plan, choose tools, and verify outputs step-by-step.
What we learned
Hands-on experience structuring production-grade Model Context Protocol transports and handling bi-directional event lifecycles. The balance between giving an agent operational autonomy and enforcing strict, deterministic safety rails around user data. How to craft concise, low-token tool definitions and system prompts that guide models toward zero-shot tool selection accuracy without unnecessary overhead.
What's next for OmniSync AI: Autonomous Multi-Device Agent & MCP Hub
Ambient Device Integration: Expanding beyond cloud APIs to connect directly into smart display interfaces, home entertainment hubs, and ambient home sensors for truly contextual awareness. Collaborative Multi-Agent Workflows: Breaking specialized jobs into cooperating sub-agents—such as an automated SRE agent, a calendar coordinator, and a personal research analyst working together under a single MCP umbrella. Local Offline MCP Runtime: Investigating hybrid edge architectures that run lightweight reasoning models locally to execute device-level tasks with zero internet latency.Ambient Device Integration: Expanding beyond cloud APIs to connect directly into smart display interfaces, home entertainment hubs, and ambient home sensors for truly contextual awareness. Collaborative Multi-Agent Workflows: Breaking specialized jobs into cooperating sub-agents—such as an automated SRE agent, a calendar coordinator, and a personal research analyst working together under a single MCP umbrella. Local Offline MCP Runtime: Investigating hybrid edge architectures that run lightweight reasoning models locally to execute device-level tasks with zero internet latency.
Log in or sign up for Devpost to join the conversation.