Hearth

Tell your home what you want in plain English. Qwen compiles it into a live deployment on $5 sensors — and reasons about what the camera sees. Edge perception, cloud brain, local-first.

Comment

Inspiration

Almost nobody automates their home — not because the hardware is expensive ($5 of ESP32 does it), but because every platform makes you program it: rules, thresholds, YAML, IFTTT chains. The intent ("warn me if the garage is left open after dark") is one sentence; turning it into a working system is an afternoon of wiring. We deleted that afternoon. Say the sentence, and the system builds itself.

What it does

Hearth is a home-automation platform where you don't write rules — you say what you want, and Qwen compiles it into a running deployment on real hardware.

  • Plug in a $5 ESP32. It self-describes what it can sense (chip temp, DHT11, HC-SR04 distance) and do (relay, servo) — no config, no firmware edits.
  • Say "if the nursery goes over 78, cut the heater and text me."
  • Qwen (qwen-plus) reads the live node registry and synthesizes the deployment — which sensors to bind, the trigger predicate, the action. You never wrote a rule.
  • At runtime, Qwen-VL (qwen-vl-plus) looks at actual camera frames and answers open-ended questions — "is this person a household member?" — that no local threshold ever could.

The result: a home that understands intent, perceives the real world, and acts on both — from a sentence.

How we built it

The honest test we held ourselves to: would this be meaningfully worse if we deleted Qwen and dropped in a 50-line script? For Hearth, emphatically yes — Qwen is load-bearing in two places a script can't touch:

  1. Authoring = program synthesis. Natural language → a real, deployed predicate, grounded in exactly what sensors exist.
  2. Runtime = open-vocabulary perception. Qwen-VL judges ambiguous situations against reference photos — no face-embedding model, no brittle threshold.

And it's deliberately token-frugal. Qwen compiles most wishes down to a local predicate that runs on the hub for free, forever, offline. The model is invoked only where open-ended reasoning is genuinely needed. That's the whole architecture: compile questions, don't stream everything to a model.

Stack (all on Alibaba Cloud, ap-southeast-1):

  • Function Compute 3.0 — the entire backend + a 21-tool Home MCP surface Qwen calls
  • Model Studio / DashScope — qwen-plus (authoring) + qwen-vl-plus (vision)
  • Tablestore — durable watches, accounts, hub pairings, device registry
  • OSS — camera frames + reference photos, served by expiring presigned URL with per-frame privacy transforms (raw | crop | downscale | redact) applied at the edge
  • ESP32 firmware with an autonomous safety-veto latch that ignores an unsafe "on" command no matter which side issued it
  • Raspberry Pi hub — node registry, mDNS re-discovery, compiled local watch runtime, on-device Qwen reasoning, offline event buffer with replay, exponential backoff, adaptive sync debounce

It's checkable. The whole cloud stack is one live call:

curl https://hearth-mcp-gqfuhlkzpo.ap-southeast-1.fcapp.run/health
→ {"ok":true,"service":"hearth-cloud","brain":"qwen","store":"tablestore","tools":21}

brain:"qwen" = a real DashScope key serving Qwen. store:"tablestore" = Alibaba Tablestore holding the data plane. tools:21 = the Home MCP surface Qwen calls. The host resolves to Alibaba Cloud LLC.

Challenges we ran into

  • Making authoring reach real hardware with zero restart. A watch you describe is Qwen-compiled, stored in Tablestore, and adopted by the hub on its next debounced device sync — running on your ESP32 about a second later. No copy-paste, no reboot.
  • Real-time without an always-on server. Function Compute scales to zero, so live dashboard readings ride a purpose-built relay while the backend, brain, and all state stay on Alibaba Cloud — best of both: instant UI, zero idle cost.
  • Surviving a $40 credit budget forced the token-frugal "compile, don't stream" design — which turned out to be the right architecture, not just the cheap one. Hearth would run the same way on a rack.

Accomplishments that we're proud of

  • The full loop is real on hardware: sensor → hub threshold eval → real HTTP actuate on the ESP32 → real phone push. Warm the board with your hand and the relay clicks.
  • Graceful offline degradation: compiled local watches keep firing with zero cloud, buffered events replay on reconnect, nodes re-find a restarted hub, and the safety veto is fully autonomous.
  • Privacy-first by construction: frames are sampled, not streamed; transformed at the edge before they ever leave the house; and served only through short-lived presigned URLs.
  • One curl proves the entire stack is live — no screenshots, no hand-waving. Everything we claim, you can hit yourself.

What we learned

  • Intent is the interface. The moment authoring became "say the sentence," every other design decision got simpler — the model, not the user, absorbs the complexity of wiring a home.
  • The best use of an LLM is often to not call it. Compiling a wish into a free, offline predicate — and reserving the model for genuine open-ended reasoning — is faster, cheaper, and more reliable than streaming everything to the cloud. Frugality and quality pointed the same direction.
  • Edge and cloud aren't a tradeoff — they're a division of labor. Perceive at the edge, reason in the cloud, act locally, and degrade gracefully when the link drops. Getting that boundary right is the whole game for physical AI.

What's next for Hearth

Deeper on-device vision so more reasoning runs on-prem, a marketplace of shareable "wishes" you can install with one tap, voice-first authoring, and expanding the self-describing node kit to cover the whole home — every device that can sense or act, wired by a sentence.

Built With

Share this project:

Updates