JARVIS (Universal Edition)

JARVIS for any business, a voice-driven multi-agent AI that tracks your metrics, handles your customers, and briefs you every morning, configured for anyone in a five-minute interview.

Comment

# Inspiration: It started with a movie frame: Tony Stark in his workshop, surrounded by floating holograms, talking to an AI that just handles things. The holograms aren't real technology — but in 2026, almost everything else in that scene is. Voice interfaces, multi-agent orchestration, and a wave of official MCP servers from real companies (RevenueCat, Buffer, Meta) meant the gap between "movie fantasy" and "buildable weekend project" had quietly collapsed.

The original version was personal: a command center for one founder's app business. Halfway through, we asked a harder question — why should JARVIS only work for us? A pottery shop owner, a newsletter writer, and a freelance consultant all have the same core problem: their metrics, customers, content, and inbox live in five different tabs, and nobody is watching them overnight. So the project pivoted from a specialized tool into a universal one: a JARVIS that any person can configure for any business in a five-minute interview.

What it does: JARVIS is a voice-driven AI operations agent. You talk to it (or text it, or message it on Telegram) and it delegates to six specialized subagents: metrics, content, campaigns, inbox, customer support, and research. It tracks whatever numbers matter to your endeavor, publishes your content, drafts replies to your customers from your own FAQ files, and has a spoken morning brief ready before you sit down. Every money-moving or outbound action is confirmation-gated — JARVIS proposes, you approve.

How we built it: Orchestration: Qwen Code on Alibaba Cloud Model Studio, using its native primitives — Markdown-defined subagents with description-based auto-delegation, Skills for the FAQ knowledge base, native Telegram Channels for the phone interface, and Cron scheduled tasks for the morning brief. A strong model (qwen3.7-max) runs the orchestrator; a fast one (qwen3.6-flash) runs subagents.

Real-world hands: live MCP connectors — RevenueCat for subscription metrics, Buffer for one-command cross-platform publishing, Meta's official Ads MCP for campaigns, and Gmail — plus built-in Computer Use for browsing. The generalization layer (the key design move): every user-specific assumption became data. A validated business.yaml profile holds who you are, what you track, and how you sound. An interview-style /setup wizard writes it for you. Metrics flow through a pluggable adapter interface — Stripe, RevenueCat, Shopify, GA4, a generic HTTP + JSONPath adapter, CSV, and manual voice logging ("log revenue 4200") — so any business whose numbers live behind any API or in any spreadsheet is covered. Voice: wake word → local Whisper STT → orchestrator → streamed ElevenLabs TTS, in a loop designed to never crash mid-conversation. Alerts: a generic KPI engine polls every configured metric and fires only on genuine threshold crossings. With threshold theta and hysteresis margin h, an "above" alert fires only when so a value hovering at the line can't spam your phone, and percentage rules use Delta 24h = fraction v t minus v t minus 24h |v t minus 24h | times 100 % with per-metric direction awareness (for revenue, up is good; for churn, up is an emergency).

Challenges we ran into: The MCP ecosystem is real but young. Meta's Ads MCP is rolling out per ad account — some accounts simply return is_ads_mcp_enabled: false, which we had to treat as a first-class state, not an error. Buffer's beta API is create-only (no edit/delete). Designing subagents that degrade gracefully around beta gates was more work than the happy path.

Generalizing without diluting. The hardest engineering wasn't any single feature — it was ripping our own business assumptions out of the architecture. The answer was the adapter pattern plus two "universal escape hatches" (HTTP + JSONPath, CSV) that cover the long tail of businesses no integration list ever will. One brain, three doors. Voice, text, and Telegram had to route into a single orchestrator entry point. Early designs accidentally created three slightly different assistants; unifying them into one orchestrator_sdk.py fixed personality drift and duplicated state. Cloud region consistency. On Model Studio, an API key, base URL, and model ID from mismatched regions fail in confusing ways — our doctor diagnostic command exists because we hit this ourselves.

Accomplishments that we're proud of: We generalized without gutting. The pivot from a single-business tool to a system any founder, freelancer, or shop owner can configure took one design move — making the Business Profile the heart of the architecture — and it worked without dumbing anything down. The five-minute /setup interview producing a fully personalized agent is the thing we'd demo first.

Six subagents, one brain, three doors. Voice, text, and Telegram all route into a single orchestrator with description-based auto-delegation across six specialized agents — and the personality stays consistent no matter which door you walk through. We shipped against real integrations, not mocks. Live MCP connectors for RevenueCat, Buffer, Meta Ads, and Gmail — including first-class handling of their beta realities (per-account rollout gates, create-only scopes) instead of pretending the happy path is the only path. An alert engine that respects your attention. Hysteresis and debounce mean a metric hovering at a threshold fires exactly one alert per genuine crossing — proven in tests that simulate crossings in both directions. No 3 a.m. spam. Safety as a feature, not a footnote. Every money-moving, outbound-email, and public-posting action is confirmation-gated by default. We're proud that handing JARVIS your ad account feels sane. Built for non-developers to actually run. MIT-licensed, a README written for humans, three example business profiles, and a doctor command that diagnoses setup problems and prints the fix.

What we learned: Verify integrations before you architect. We checked every MCP server's real, current capabilities before writing a line of design — and the betas' limits (write gates, create-only scopes) reshaped the architecture. Assumptions here would have sunk the build.

Configuration is a product feature. The /setup interview matters as much as the agents. If a non-developer can't get to a working JARVIS, the generalization is fictional. Safety gates build trust. Paused-by-default campaigns and approval-gated emails weren't compliance chores — they're what makes it psychologically possible to hand an AI your ad account.

What's next for JARVIS (Universal Edition): A hosted multi-tenant version (one deployment, many users, isolated data), more adapters driven by real user requests, and richer two-way voice. The holograms, sadly, remain out of scope.

Built With

Share this project:

Updates