CompanionAgent

Inspiration

We wanted to fix a specific annoyance with generic chatbots: they treat every question — a budgeting question, a health worry, a grocery run — with the same flat, undifferentiated voice. Real life doesn't work in one category at a time either. "Pick up my prescription and plan tonight's dinner party" is two different problems wearing one sentence. We wanted an assistant that could actually recognize that and hand each piece to something built for it, then hand back one coherent answer instead of two disconnected ones.

What it does

Everyday Agents is a personal assistant that routes each message to one or more specialized agents — Money, Health, Home, Errands, Family, and a general Daily Life agent — based on the intent behind it. If a message touches multiple domains, the relevant agents run in parallel and their responses are blended into a single natural reply, rather than being shown as separate answers stapled together. It supports text, voice, and image input (for example, snapping a photo of a receipt), and streams responses back token-by-token over a WebSocket so replies feel immediate.

Anything that looks like a consequential action — a financial transfer, canceling a subscription — doesn't execute automatically. It surfaces an approval modal first, so the assistant stays useful without acting unsupervised.

How we built it

The backend is a FastAPI service with a router/orchestrator at its core: it classifies intent, dispatches to the right agent(s), and merges their outputs. Each agent is a distinct persona layered on Google Gemini Pro, with its own system prompt and a slice of conversation memory scoped to its domain, persisted in SQLite. The frontend is React 19 with Vite, styled in Tailwind and animated with Framer Motion, using a useReducer + Context store instead of a heavier state library. The client talks to the backend primarily over WebSockets for streaming, with a REST fallback and exponential-backoff reconnection if the socket drops.

Challenges we ran into Blending, not stacking: getting multiple agents' responses to merge into one voice — rather than reading like two bots taking turns — took real prompt-engineering iteration. Keeping memory scoped without losing context: making sure the Money agent's history doesn't bleed into a Health conversation, while still letting the router see enough shared context to blend coherently. Streaming reliability: building reconnection logic that gracefully falls back to REST without the user noticing a hiccup in the chat. Safety without friction: deciding where the line sits between "just do it" and "ask first" for agent-proposed actions, so approvals feel like a safeguard and not an annoyance. What we learned

Multi-agent orchestration is less about the individual agents and more about the seams between them — the router's classification logic and the blender's merge step ended up being where most of the actual engineering effort went, not the agent prompts themselves.

What's next Smarter intent classification (moving beyond keyword-style routing toward more robust classification) Deeper cross-agent context sharing for multi-domain requests More granular approval rules per agent/action type macOS/Linux setup parity (currently PowerShell/Windows-first)