Ellie

Normal reminders remember a time. Ellie remembers your home.

Comment

Inspiration

We are always occupied with something in our hands, and it is easy to forget household tasks. I see my mom do this too, so I wanted a household solution that remembers for us.

Alexa can already remind us about a date we choose. But household maintenance needs more context: when a particular gas cylinder will run out, whether a purifier problem happened before, which repair fixed it, whether something is under warranty, and what regional booking rules apply.

That led to Ellie: a household memory agent that turns everyday home history into useful, explainable action.

What it does

Ellie remembers household events such as supplies being refilled, appliances being serviced, problems being reported, bookings, deliveries, and repairs.

It then:

  • Predicts upcoming maintenance from the household’s own history.
  • Applies sourced regional rules, units, vocabulary, providers, and safety contacts.
  • Diagnoses problems using previous events, repairs, warranty information, and cited evidence.
  • Lets one sentence trigger multiple tools—for example, logging a cylinder change and diagnosing a beeping purifier.
  • Sends detailed repair instructions and booking links to the phone.
  • Records confirmed fixes and recomputes future predictions.
  • Stops unsafe DIY guidance when the user reports a dangerous situation.
  • Lets users inspect, export, withdraw consent, forget an item, or erase their household data. For example, a user can say:

“The cylinder’s changed and the purifier is beeping again.”

Ellie logs the event, diagnoses the purifier, updates the prediction, and can send the detailed fix guide to the phone.

How we built it

Ellie is built around a real self-hosted MCP server using the MCP SDK v1 and Streamable HTTP.

The architecture is:

Alexa+ simulator or MCP host | v Streamable HTTP MCP server | v Typed MCP tools | v Domain logic

  • Household memory
  • Prediction engine
  • Diagnosis and safety
  • Phone handoff
  • Privacy and consent | v Adapters
  • File storage or DynamoDB
  • Mock LLM or Amazon Bedrock
  • Mock notifier or Amazon SNS
  • Local scheduler or EventBridge Scheduler

The simulator communicates with the server through a real MCP client. It does not import the domain logic into the interaction path.

The server exposes typed tools for logging events, reading history, predicting due dates, diagnosing problems, sending phone handoffs, managing consent, and performing confirmed privacy actions.

The project runs locally with an empty .env using mock and file drivers. AWS-backed adapters are available for DynamoDB, Bedrock, SNS, EventBridge Scheduler, and SQS.

Facts such as dates, due status, warranty status, and safety rules are computed by the server. The optional language model is restricted to ranking server-generated diagnosis candidates and explaining supplied evidence.

Challenges we ran into

The Alexa+ MCP Toolkit is partner-gated, so we could not rely on a production Alexa+ connection for the demonstration. We built a clearly labelled browser simulator that exercises the real MCP server over Streamable HTTP.

Another challenge was making diagnosis useful without allowing a model to invent household facts. We solved this by computing safety classification, evidence, dates, and candidate causes deterministically before any model call.

We also had to design safe phone handoffs. Messages require consent, phone links are signed and time-limited, contacts are masked, and destructive privacy actions require explicit confirmation.

During testing, we found a privacy bug where a numeric-looking confirmation identifier could be mistaken for a phone number by a redaction heuristic. That could make a deletion confirmation fail. We fixed the issue and added regression coverage.

We also documented platform friction, including differences between Alexa+ authentication expectations and standard MCP authorization behavior.

Accomplishments that we're proud of

  • Built a real MCP server using the MCP SDK v1 and Streamable HTTP.
  • Created a simulator that communicates through a real MCP client.
  • Demonstrated two MCP tools from one spoken sentence.
  • Built household-history-based prediction instead of relying only on generic intervals.
  • Added deterministic diagnosis evidence and safety gates.
  • Created a voice-to-phone experience for repair guides and bookings.
  • Added regional packs for different terminology, units, providers, and safety rules.
  • Added privacy inventory, consent management, export, item deletion, and household deletion.
  • Added mock adapters so the complete demo works without AWS credentials.
  • Verified the server with MCP Inspector and automated tests.
  • Kept the voice responses short while moving detailed content to visual cards and the phone.

What we learned

We learned that household memory becomes much more valuable when it connects past events to future action.

Remembering that a filter was changed is useful. Remembering how often it is changed, predicting the next date, checking the warranty, explaining the evidence, and offering the next action is much more useful.

We also learned that safety and privacy must be part of the architecture from the beginning. They cannot be added as a final layer after the main assistant experience is complete.

Finally, voice, cards, and phone screens should each have a different job:

  • Voice should be concise and conversational.
  • Cards should provide glanceable context.
  • The phone should carry detailed instructions, links, chat, and confirmations.

What's next for Ellie

Next, we plan to:

  • Verify and expand regional facts using official sources.
  • Add more household categories and service-provider integrations.
  • Complete native-speaker review for additional phone languages.
  • Measure live AWS latency and operational behavior.
  • Add more integrations for receipts, warranties, and maintenance providers.
  • Connect the MCP server to supported Alexa+ environments when access is available.

Ellie’s goal is simple:

Turn household memory into useful, explainable action while keeping control with the people who live there

Built With

Share this project:

Updates

Submission history