EWOK | Devpost

Home page
FinchNode dashboard for health data
Nessie data for financial dashboard

Inspiration

Inspired by the idea that someone is yet to build "AI for a billion users", we kept coming back to the same picture: a parent or grandparent who will not install another app, who talks slowly, who pauses a long time, and who is exactly the person scammers go after. Most “AI assistants” assume a fast speaker, a clean microphone, and a willingness to tap through five confirmation screens. EWOK started as: what if the phone was the product? You dial, you talk, it handles email and the computer, and money or identity never happens quietly. MHacks was the excuse to make that real instead of a slide. We wanted something an older adult could use themselves large type, one thing at a time, memory they can read and delete not a caregiver dashboard with the older person as a data source.

What we learned

Voice is not “chat plus a microphone.” Silence feels like a crash, so we spent a lot of time trying to optimize for the fastest TTFT. We learned to stream speech, speak fillers while tools run, and tune turn-taking for long pauses. We learned that putting rules inside the model is how you get talked out of them; spending caps, scam flags, and “say the actual dollar amount before you pay” had to live outside the LLM. Especially when integrating computer use, which typically fills up the models context and takes up so much time, so we had to create custom caching for the memory between speech and computer use so the slower computer use context windows would not take over

How we built it

The brain is a Python WebSocket gateway (claw) on Pydantic AI: sessions, tools (memory, web search, computer use, automations), and streamed events. The browser dashboard is the same agent — chat, voice (ElevenLabs STT/TTS), and panels for mock bank (Nessie) and synthetic health (FinchNode). Phone is Vapi: telephony and speech on their side, custom LLM URL into our gateway so persona, memory, and policy stay in one place. A small Node layer handles ElevenLabs webhooks, scam checks, and notify. Policy is not a prompt; it is code that can refuse. Memory is files you can open: MEMORY.md, daily notes, identity. Demo data is labeled mock on purpose.

EWOK is not one process. A person can be on a phone call, in the browser, and have a scheduled reminder due at the same time. “Take the morning pills” or “check the bank” must fire once, even if two gateways, a cron watcher, and a channel adapter all wake up in the same second. A normal stack would bolt on Redis, a lock table, and a worker pool, then hope the locks and the job rows stay in sync.

We put that coordination in SpacetimeDB instead, because it is not “Postgres plus a message bus.” Our job table stores OpenClaw-shaped automations (schedule, payload, status) plus a worker lease (claimed_by, claimed_at_ms). Claiming work is is one reducer, claim_due_job, that finds a due pending job and moves it to running for a specific worker in a single transaction. Two workers cannot both win. Completing, rescheduling, canceling, and reclaiming stale leases are reducers too (complete_job, reclaim_stale, …). The invariant lives in the database, not in each Python process hoping it was polite

Challenges

Our biggest challenge came from not forking an existing harness like OpenClaw, but instead building the harness from scratch using the Pydantic AI agent library. That meant re-deriving the control plane ourselves: WebSocket connect handshake, req/res/event frames, immediate agent accept with a runId, agent.wait, per-session lanes and queue modes, MEMORY.md bootstrap, automations that the gateway actually fires. The agent loop is Pydantic AI (typed deps, tools, streaming events) plus the Harness capabilities for workspace and computer use. Compatible feel, not a clone: no protocol v4, no messaging-platform identity. Every time we thought “OpenClaw already does this,” we had to decide whether to copy the idea or the implementation.

We chose to do this mostly due to the safeguards needed for having a non technical person have an AI agent control their lives. The original OpenClaw harness has complete hands off and would allow the agent to build new skills, but it would also mean the program gains more potential vulnerabilities, and a changing interface could become confusing for people. Additionally, the need to vet a potentially infinite number of vulnerabilities the agent could find on the internet through third party skills makes the overhead of running this tool too high for a non-technical user