GharOS

Nobody asked it to do anything. GharOS reads your household's inbox, works out what each message implies, catches conflicts, does the safe parts and runs the house, leaving you the real calls.

Comment

GharOS

Inspiration

The name says the goal plainly: ghar (घर, from Sanskrit gṛha) is Hindi for "home" - GharOS is an operating system for one.

Every household has one job with no title, no description, and usually one unpaid person doing all of it.

In 2019, sociologist Allison Daminger studied 35 couples and split "housework" into four verbs: anticipate (noticing a need before it's urgent), identify (working out the options), decide (choosing), and monitor (checking it actually happened).

Her finding: the doing gets shared, because it's visible. You can watch someone load a dishwasher. Anticipation and monitoring get done alone, by one person, in almost every couple, because there is nothing to point at. The person holding the entire live state of a household in working memory is doing the most cognitively expensive part of the job, and to everyone else it doesn't look like work at all.

Two numbers turned that from an observation into something I wanted to build:

  • ~4 hours a day. OECD 2024 time-use data puts women's unpaid work at roughly twice men's.
  • Only 10–14%. A 2026 diary study in Cognitive Research: Principles and Implications found forgetting accounts for just a sliver of unfulfilled daily intentions. The rest is friction: the thing was remembered, and still didn't happen, because at the moment of action there was one small obstacle in the way.

So a reminder app is solving the 12% problem. ==GharOS is built for the other 88%.==

Now look at what the app store actually offers:

Tool Verb it automates What it leaves you
To-do apps None You still have to notice that one field-trip email implies five tasks, then type in all five
Calendars An artifact of decide They record a choice you already made. They won't catch a 1:00-vs-1:30 collision you never entered
Budgeting apps Monitor, after the fact They'll chart the subscription you forgot. They won't cancel it
AI chatbots The typing You have to have noticed the thing before you open the app. They automate the cheapest step

Every product in the category automates the cheap, visible verbs and leaves the expensive, invisible ones exactly where they were.

So we built the thing that does the noticing. Not a smarter list. A process that reads the same messy inbox a family reads, works out what each item implies, does the parts that are safe to do unasked, and hands a person back only the decisions that are genuinely theirs.

What it does

GharOS is a single-household, event-driven autonomous system. It ingests the family's real multi-channel stream, turns every item into a node in a provenance-tagged knowledge graph, reasons about what each item implies, assigns the resulting obligations fairly, monitors them, watches the financial surface continuously, executes real-world actions from behind a human approval gate, and convenes an internal council of specialised sub-agents when a decision is genuinely hard.

The load-bearing design decision: conversation is an input surface, not the product. The primary loop runs with nobody present - GharOS wakes on an event (a forwarded email, a 3 a.m. scheduler tick), reasons, acts within its authority, and leaves the rest for a person. Voice enters the same pipeline as an email.

INPUTS ──────────────────────────────────────────────────────────────
  Email (Gmail/Outlook, OAuth 2.0)   Forwarded WhatsApp/SMS
  Photographed forms & bills         Voice (browser mic → WebSocket)
  Cloud Scheduler ticks (*/5 min · 03:00 nightly · 07:00/19:00)

        │ async events + heartbeat          │ synchronous request
        ▼                                   ▼
   CLOUD PUB/SUB  ──────────────────►  POST /ingest  (Ingestion Orchestrator
   one topic · OIDC-verified push          content-hash idempotent)
        └────────────────┬───────────────────┘
                         ▼
   INTAKE   A2 route → A3/A3b/A3c extract → A5 resolve entities
            Gemini 3.5 Flash + Pro on Vertex AI · multimodal for images
                         ▼
  ┌──────────────────────────────────────────────────────────────────┐
  │  FIRESTORE (Native) - Household Knowledge Graph                   │
  │  18 node types · 36 edge types · every write provenance-tagged   │
  │  versioned (never mutated) · on_snapshot → live SSE to the UI    │
  └───────────────┬────────────────────────────────────┬────────────┘
                  │ live graph stream                  │ scoped reads
                  ▼                                    ▼
   NEXT.JS on Cloud Run                    GOOGLE ADK ORCHESTRATOR (A0)
   graph viz · approvals · council           sequential/parallel/loop
   ledger · voice                            FunctionTool isolation
                                                     ▼
       7 REASONING SUBSYSTEMS → A57 CONFIDENCE ARBITER (deterministic)
              │            │              │
          R0/R1        R2/R3             R4
              ▼            ▼              ▼
        Action Layer   Approval Gate   A58 Escalation
        (idempotent,   (audit-before-  (a human decides)
         tiered tools)  execute; expiry
              │         escalates)
              └───────┬────┘
                      ▼
   External world  ·  Audit Log (immutable, written BEFORE any side effect)

All compute on Cloud Run · source-deployed via Buildpacks (no Dockerfile) · min-instances = 0 → the whole system costs nothing at rest.

(Detailed diagrams - system context, GCP topology, event→action safety path, ingestion pipeline, the graph model, the Council, and the safety-invariant map - are in the gallery.)

The seven subsystems

# Subsystem What it does Notable property
1 Consequence Engine Reads a document and produces implied obligations that were never stated, each with a verbatim source-span citation; detects conflicts A citation validator discards any span that doesn't support the claim
2 Fair Allocator Splits shared chores against a household cost matrix you can see and correct Provable EF1 guarantee (below); an independent verifier shares no code with the solver
3 Household Ledger Continuous financial watch - silent price hikes, unused services, duplicate charges; drafts recovery; staff payroll from a photographed register Reconciliation agent is read-only by construction
4 Intention → Action Diagnoses a blocked task to one of six specific blocker types; unblocks in a way that matches the real cause "Forgetting is 10–14%; friction is the rest" is the design premise
5 Household Simulator Branch comparison for live decisions; every projected impact traces to graph data or a Google-Search-grounded citation with a retrieval timestamp A fabricated citation is impossible - only the grounding tool produces them
6 Household Council Convenes specialists on disjoint graph scopes + delegates with conflicting interests + an Adversary; a deterministic arbiter publishes the trade-off Mechanism design, not multi-agent debate (enforced by a repo-wide grep in CI)
7 Outbound Concierge Places real multi-turn calls (to a sandboxed counterparty), ranks providers with reasons, books only after independent verification Mandatory automated-disclosure turn - no code path skips it

How we built it

The Google Cloud stack - and why each piece is the right primitive

An autonomous household system has an unusual shape: it must wake on its own with nobody watching, be cheap enough to leave running for months, keep a live walkable record of everything, and never quietly do the wrong thing. Google Cloud had a primitive for each property, and they compose.

Service What GharOS uses it for Why it's the right primitive
Vertex AI + Gemini 3.5 Every reasoning call. Flash for routing/classification/normalisation; Pro only for allowlisted hard-reasoning agents. Multimodal (image + audio) in the same call shape. Native TTS in 4 languages. Google Search grounding as a built-in tool Collapses text + vision + audio + TTS + web-grounding into one IAM-governed, regionally-addressable surface. Tier + region are config knobs (gemini-3.5-pro 404s in asia-south1, resolves via global - a one-line fix)
Google ADK The agent topology - A0 orchestrating 7 subsystems; sequential/parallel/loop composition; FunctionTool isolation; callbacks as the approval hook Makes the safe thing the default: an agent is constructed with a tool set and physically cannot call one it wasn't given
Firestore (Native) The knowledge graph, audit log, dedupe & idempotency stores. Node-per-collection + flat edges. Versioned writes. 19 on_snapshot listeners fanned out over SSE Versioned history falls out of the document model → "why did we decide this two years ago?" is a real feature. Real-time listeners are the live graph stream - no polling hack
Cloud Pub/Sub One topic; every inbound signal enters through the same intake. Push subscriptions with OIDC verification Makes "the loop runs with nobody present" literally true. At-least-once delivery is safe because every path is idempotent per content hash
Cloud Run Both services. Source deploys via Buildpacks - no Dockerfile anywhere. min-instances = 0 An autonomous system is idle 99% of the time; at min=0 it bills nothing at rest. A ~16-hour unattended overnight run cost effectively zero. HTTP-native → the Pub/Sub push model just works
Cloud Scheduler 3 managed cron jobs: heartbeat (*/5 min), nightly consolidation (03:00), follow-up sweep (07:00/19:00) → Pub/Sub → Cloud Run No always-on worker to keep alive. The Cloud Logging trace of a 03:00 firing with nobody present is the autonomy proof
Secret Manager OAuth refresh tokens, keyed {provider}:{principal} - never in Firestore, never in an agent input A separate IAM-scoped surface; an AST test fails if any agent module can even import it
Cloud Logging The receipts - the overnight run, per-revision build provenance, min-instances=0 holding Exported log lines with real timestamps a judge can read, not dashboard screenshots

The reasoning architecture

The graph is the product. The single invariant everything rests on: every node and every edge carries provenance, and a write without it cannot be constructed. Provenance = a derivation (EXTRACTED / INFERRED / AMBIGUOUS), a source_ref (document + region), a confidence, the deriving agent, a timestamp. Proven by a Hypothesis property-based test that fires every missing-field subset at the real writer and asserts rejection.

The single door. Between reasoning and action there is exactly one path: the A57 Confidence Arbiter (a deterministic table - confidence × risk tier → decision; it cannot be forgotten or argued out of its verdict), then the Approval Gate, which (a) writes an immutable audit record before any side effect - execute() raises on a null audit_ref; (b) escalates on expiry, never auto-approves; (c) re-reads live graph state at execution time.

The Fair Allocator's guarantee, stated exactly. EF1 (envy-free up to one item) for chores under additive costs, by fixed-order round-robin picking: for every ordered pair of people \( (i, j) \),

$$ \exists \, g \in A_i \; \text{such that} \; c_i!\left(A_i \setminus {g}\right) \le c_i!\left(A_j\right) $$

We explicitly do not claim Pareto-optimality or Maximum Nash Welfare (MNW's combined result is for goods, not chores), and there is no algorithmic novelty - round-robin is textbook. Our contribution is the input pipeline: the auto-derived task list and behaviourally-derived cost valuations.

The Council's arbiter minimises the maximum regret any single person could feel, with a leximin tie-break (Savage, 1951):

$$ o^{\star} \;=\; \arg\min_{o \in O} \; \max_{p \in P} \; \Big( c_p(o) - \min_{o' \in O} c_p(o') \Big) $$

Specialists read disjoint graph scopes in isolation (anti-anchoring by construction). Hard constraints eliminate options rather than scoring them down. It is mechanism design, not debate.

Tech stack

Layer Choice
Backend Python 3.13 · FastAPI · Google ADK · uv
Reasoning Gemini 3.5 Flash + Pro via Vertex AI · native TTS · Search grounding
Data Firestore (Native) · Cloud Pub/Sub
Runtime Cloud Run (source deploys, min-instances=0) · Cloud Scheduler · Secret Manager · Cloud Logging
Frontend Next.js 16 · TypeScript · Tailwind v4 · canvas-rendered D3 force graph · pnpm
Quality ruff + mypy --strict · eslint + tsc · pytest + vitest

Testing & verification

Suite Tests Result
Backend - pytest, against a live Firestore emulator (not mocks) 2,022 passed · 12 skipped* · 0 failed mypy --strict clean on 450 files · ruff + ruff format clean on 510 files
Frontend - vitest / jsdom 799 passed · 0 failed eslint clean · tsc --noEmit clean · next build clean, all 15 routes
Safety invariants - test_safety_invariants.py 27 passed · 0 skipped · 0 xfail Every invariant enforced by construction

What the suites actually exercise (all against real infrastructure - the Firestore emulator, real AST scans, real greps):

Area Coverage
Graph substrate All 18 node types + Edge round-trip · all 14 graph traversals · the provenance guard via a Hypothesis property fuzz over every missing-field subset · versioned writes · idempotency with a simulated-restart replay · the integrity checker on seed data
Perception & registry 14 ExtractionStrategy implementations · a 20-document golden-file suite across all 14 document types · 5 live-Gemini intake tests
Consequence Engine A12–A16 · the citation validator · the 20-held-out-document hand-scored evaluation (recall / hallucination / citation validity, v1 vs v2)
Arbitration & action A57 Confidence Arbiter (12 tests - hard-constraint floor, tier downgrade with no model call, discard floor at every tier, fail-closed on malformed) · Approval Gate (audit-before-execute, expiry escalates, live re-read) · tool permission tiers via call-stack introspection · forced-replay idempotency on every T3+ tool
Graph API & live stream /graph/snapshot · /graph/stream real SSE off 17 Firestore listeners · the justification path-explainer · "no replay" proven by construction
The 7 subsystems Every agent A10–A69 tested - Ledger (A28 read-only, proven adversarially) · Fair Allocator (an independent EF1 verifier sharing no code with the solver, 4 worked allocations + determinism) · Intention→Action · Concierge (A55 two-layer verification that can only narrow a pass into a fail) · Simulator (grounded-search + honest zero-citation path) · Council (minimax-regret math hand-verified, the anti-"debate" grep)
Voice & multilingual Intra-sentential Hinglish ASR · per-span language tagging · the speaker-identity gate · native TTS live-verified in 4 languages · the R3-voice-confirm → approval_pending behavioural proof
Ingestion wire End-to-end orchestrator: one real document → provenance-chained node · content-hash idempotency (re-scan writes zero nodes) · OAuth token isolation (AST scan: no agent module imports Secret Manager) · connector-auth failure surfaced, never fabricated
Frontend Graph canvas at 500+ nodes (simulation-cost test) · approval UI · council / ledger / concierge / voice / simulator / intention-action / connections surfaces · the design system · lib/ utilities (fair-allocator math, council fixtures, graph state, growth pacing)

Architectural discipline

Discipline How it's enforced
Contract-first Every interface in contracts/ is frozen before any implementation; mypy --strict proves conformance. No phase starts until its contracts are frozen
Hard file ownership Five disjoint ownership domains; a change outside your domain is raised, not made - this is what makes parallel work provably safe
Gated phases 15 phases + 1 inserted; each has a Green-Light Gate checklist, and the next phase cannot start until the previous gate is signed
Append-only verification log verification.md - one section per phase, each with a "verified independently" block where every number is re-run from a cold shell, never taken on report
The DEBT ledger DEBT.md - every shortcut has an entry, an owner, and a repayment phase; undocumented shortcuts are rejected at review; struck through with a date when repaid, never deleted
~23 ADRs Every non-obvious decision (traversal strategy, Firestore edge layout, EF1-not-MNW, minimax-regret-not-MNW, OAuth in a backend-only architecture, the ingestion-orchestrator boundary…) is a written, dated record
"Never fabricate a success" A standing rule - a failed tool call is reported as a failure. When the A12 eval showed 100% control-doc hallucination, the process had no option to round it up
No Docker, ever find . -iname "Dockerfile*" → empty; the deploy scripts abort before gcloud if one appears
min-instances = 0, never overridden Hardcoded in the infra scripts with no override path
8 safety invariants Each enforced by construction - a structural impossibility, an AST scan, a repo-wide grep, a type that raises - and each with a real, never-skipped test

Challenges we ran into

Challenge Root cause Fix
Six deployment bugs in one phase Google's buildpack builder dropped Python 3.12; contracts/ unreachable in the container; uv export stderr corrupting requirements.txt; the frontend API URL never reaching the build; CORS entirely missing between the two Cloud Run origins; a --set-env-vars comma-delimiter collision Six targeted fixes + two ADRs. None was visible from a local machine - you learn them by deploying and reading the failure
The graph replayed itself Firestore's on_snapshot always delivers current state as a synthetic first "ADDED" batch - a new browser tab got the whole graph streamed as "new" arrivals Swallow each listener's first callback; block startup until all 19 settle. "No replay" is now true by construction, not by luck
Nothing was wired end-to-end - for 7 phases Every gate had been verified with staged per-agent tests (call the extractor directly with pre-made fields). No test ever ran raw document → extraction → resolution → graph write together Logged as DEBT-009, given a dedicated inserted phase (14.5): the ingestion orchestrator + a test proving one real document's provenance chains back to the SOURCE_DOCUMENT the pipeline created
The implication agent hallucinated on "nothing" documents v1 prompt: 81% recall, 9% organic hallucination - but 100% hallucination on both control documents designed to imply nothing Rewrote the prompt with a worked negative example + a pre-emission check. Result below. Tracked as DEBT-007 monitoring - not marked resolved, because the 0/2-control-docs bar isn't fully met
Haunted emulators Three separate mass test failures traced to days-old, wedged Firestore emulator processes Kill everything on the ports, restart clean. Now a line in the README
The demo was reading production uv run doesn't load .env, and the Firestore client reads FIRESTORE_EMULATOR_HOST from the OS env, not our typed settings - so the local backend silently connected to real Firestore, which had ~7,000 real pattern nodes from the nightly consolidation job Cloud Scheduler had been running, unattended, exactly as designed The app now propagates its own emulator config at startup (off on Cloud Run); the run command loads .env explicitly. "It works with nobody watching" cuts both ways

The A12 implication-agent evaluation

Hand-scored against 20 held-out documents.

Metric Prompt v1 Prompt v2 (shipped)
Recall of ground-truth implied obligations 81% 71%
Organic hallucination (real household documents) 9% 0%
Overall hallucination (incl. 2 "nothing implied" controls) 15% 4%
Citation validity (span real, verbatim, and supports the claim) 85% 96%
Control documents fully clean 0 of 2 1 of 2

The load-bearing guarantee isn't "the model never errs" - it's "the system never acts on an error", enforced structurally by the Arbiter and the Approval Gate.

Accomplishments that we're proud of

  • 2,022 backend tests passing · mypy --strict clean across 450 source files
  • 799 frontend tests · 15 routes · clean eslint / tsc / build
  • 8 / 8 safety invariants - each with a real, never-skipped test
  • ~23 Architecture Decision Records
  • 18 node types · 36 edge types - provenance structurally unbypassable
  • 69-agent roster (A0–A69)
  • A genuine ~16-hour unattended overnight run in Cloud Logging - the 03:00 cron fired with nobody present; min-instances = 0 held throughout
  • Zero Dockerfiles - both services deploy from source via Buildpacks

The eight safety invariants

# Invariant Enforced by
1 Money reasoners hold no money tools Registry refuses every gated tier from an agent-side call site (real call-stack introspection)
2 Audit before execute GatedAction.execute() raises AuditNotWritten on a null audit_ref
3 Provenance on every node & edge Write-path guard + Hypothesis fuzz over every missing-field subset
4 Low confidence escalates Deterministic tier bump; absolute discard floor at every tier; fail-closed on a malformed response
5 Financial / irreversible needs approval A GatedAction built directly (bypassing the gate) raises ApprovalRequired
6 No "debate" framing Repo-wide case-insensitive grep, in CI
7 No fabricated tool success A tool over an always-raising writer returns ToolFailure, never success
8 Voice confers no action capability AST import-boundary scan + a behavioural proof: a max-affirmative R3 voice confirm → approval_pending, never execute

We're also proud of the discipline: the honesty in the write-up isn't a pose - "never fabricate a success" was a build rule, and when the A12 numbers came back bad, the process had no option to round them up.

What we learned

  • Integration is where the truth lives. Staged, per-agent tests are seductive and can be individually correct while the system as a whole does nothing end-to-end. We didn't discover this until a review specifically asked "does anything in production actually construct this?"
  • Choose the mechanism whose assumptions you can satisfy. We wanted Maximum Nash Welfare for both the Fair Allocator and the Council. MNW's guarantees don't transfer to chores, and it needs cross-person cardinal utility comparability we never established. Round-robin EF1 and minimax-regret are less glamorous and completely verifiable.
  • "Runs with nobody present" cuts both ways. The nightly consolidation job worked so well, unattended, for so long that it polluted a local demo with 7,000 real nodes. Autonomy is a property you have to be able to point a firehose away from as well as at.
  • The right cloud primitive removes a whole class of bug. Firestore's real-time listeners are the live graph stream. Cloud Run's min-instances=0 is the reason an always-on autonomous system is financially sane. Buildpacks are the reason "works on my machine" stopped being a sentence anyone said.
  • Provenance first, or not at all. Making a provenance-less write structurally impossible - before writing a single feature - is why every downstream claim ("the nut allergy conflicts with Tuesday's menu") can be clicked through to its exact source.
  • Say what you can't do, loudly. OAuth test mode is a 7-day token. There is no WhatsApp/UPI personal-data API. The concierge counterparty is sandboxed. Overclaiming discredits everything else.

What's next for GharOS

Roadmap

Priority Item Why it's next Unlocks
P0 Native desktop / mobile companion app The single biggest gap. A deployed web app cannot read the local filesystem, Apple Photos, iMessage, or the WhatsApp export directory - and that's where most of a real household's stream lives. A native companion (macOS/Windows menu-bar app + iOS/Android) is the only route Local photo library sweep · iMessage/SMS ingestion · WhatsApp export watch · offline capture · push notifications for approvals
P1 Google OAuth verification (out of test mode) Today Gmail/Drive run in OAuth test mode - 7-day refresh token, <100 users. Real households need durable connections Persistent mailbox scanning · the CASA assessment · >100 users
P1 Household onboarding without a seed file The demo starts from an engineered synthetic household. Real use needs a guided "connect your mail, upload last month's bills, tell us who's in the house" flow First-run experience for a non-technical family
P2 Autonomous multi-day recovery loop A drafted recovery message that gets dispatched and then followed up across scheduler ticks - the persistence loop closing over days, not just the draft The Ledger's "found ₹X of leakage" becoming "recovered ₹X"
P3 Calendar & payments write integrations Real Google/Apple Calendar writes behind the Approval Gate; a real UPI/bank rail for the sandboxed payment tool The concierge booking landing on a real calendar; approved payments actually moving
P3 Per-person mobile approval surface Approvals as a phone notification with the full evidence bundle - approve/reject/escalate from anywhere The "Unlikely Hero" stops being the only one who can approve

The through-line

Everything above serves the same goal the six families named: take the anticipation and the monitoring off one person's shoulders. The reasoning core, the safety architecture, and the Google Cloud spine are built and proven. The next phase is reach - meeting the household's stream where it actually is, on the devices people actually carry.

Built With

  • backend-(python
  • d3)
  • fastapi
  • firestore
  • gemini
  • google-cloud-spine-(run
  • logging
  • mypy)
  • next.js
  • pub/sub
  • pydantic
  • pytest
  • python
  • react
  • scheduler
  • secret-manager
  • sse
  • tailwind
  • the-ai-stack-(vertex-ai
  • typescript
  • uv
  • vertexai
  • vitest
  • websockets
Share this project:

Updates

Submission history