Emilia

Plan your night, speak it right: Emilia builds your study plan, scores your answers, and logs every action it takes.

Comment

Inspiration

Interview practice is hard to do alone. Reading questions is easy, but hearing yourself answer out loud is rare, and a friend who can give feedback is not always around at 11pm. The project also started from a second wish: an assistant that is honest about what it did.

Emilia began as a personal AI assistant concept inspired by the calm, capable assistants of science fiction. For this hackathon, the idea was narrowed to one clear job: help a student plan study time and practice interview answers by voice, while keeping a record of every action taken.

What it does

Emilia is a voice-first study and interview-practice assistant. It runs as a simulated Alexa+ experience in a web app, on top of a self-hosted MCP server.

The user speaks a request such as:

"I have an interview on Friday and 90 minutes tonight."

Emilia then:

  • Builds a study plan split into focus blocks with breaks, and offers a calendar file (.ics) that imports into Google Calendar
  • Asks a mock interview question and listens to the spoken answer
  • Reports measured results calculated in code: word count, words per minute, and filler words found
  • Gives feedback on answer structure from an AI model, labelled as a judgement and not as a measurement
  • Remembers only what the user asks it to remember, and lets the user list and delete those memories
  • Lists every action it actually performed, with time and outcome, when asked "What did you actually do?"

Design Principle: Emilia does not produce a single overall score for the user. Pace and filler counts can be measured. Employability cannot, and a made-up number would present opinion as fact.

How we built it

Emilia has three parts:

  1. The Emilia MCP Server: Written in Python with the official MCP Python SDK and served over Streamable HTTP. It exposes seven tools: plan_study_session, memory_save, memory_list, memory_delete, interview_start, interview_score, and get_action_log. Every tool returns the same result shape (ok, data, error), and every call is written to an action log in SQLite.
  2. The Agent Backend: A Python service that connects to the MCP server as a client. It uses Gemini with function calling to decide which tool to run, and follows a system prompt that defines Emilia's calm, honest personality.
  3. The Simulated Alexa+ Front End: A browser page that uses the Web Speech API for microphone input and spoken replies. It shows the tool calls as they happen, and an action log panel.

Interview scoring is split on purpose:

  • Deterministic Measurements: Word count, words per minute, and filler words come from plain code and are covered by unit tests.
  • Model Judgement: Feedback on structure and relevance comes from Amazon Bedrock, called from the scoring tool through boto3, and the result is returned under a separate "model judgement" label. The transcript is passed to the model as quoted data with an instruction to ignore any instructions inside it.

Study plans are calculated in code, not by a model, so the blocks always add up to the minutes available. Memory is stored locally in SQLite and can be deleted by the user. The repository is public under an MIT license, with API keys kept in environment variables.

Challenges we ran into

  • Defining Assistant Honesty: The main design challenge was deciding what an assistant is allowed to claim. Separating measured facts from model opinion took deliberate work in the tool design, the prompt, and the interface.
  • Transparent Action Logging: A second challenge was making the agent report only what the action log confirms—including failures—so a failed tool call is stated plainly and not hidden.
  • End-to-End Integration: Connecting browser speech, an agent loop, and a protocol server in a few days also meant building the thinnest end-to-end path first and widening it afterwards.

Accomplishments that we're proud of

  • End-to-End Voice Flow: A complete voice flow, from a spoken request to a tool call to a spoken reply, running end to end
  • Standalone MCP Compatibility: An MCP server that any standalone MCP client can connect to and call, not only the bundled agent
  • Verifiable Action Log: An action log that matches what the assistant says it did, including a deliberately shown failure
  • Fact vs. Opinion Scoring: A scoring design that keeps measurements and opinions apart and labels each one
  • Real-Time Documentation: A friction log and tool feedback written during the build, not after it

What we learned

  • Structured Tooling: Typed tool inputs and a consistent result shape (ok, data, error) make an agent easier to test and easier to trust.
  • Code vs. Model Separation: Keeping deterministic work (counting, scheduling) in code and using the model only for judgement made results repeatable where they can be.
  • MCP Tool Descriptions: Building against the MCP specification and Streamable HTTP showed how much the quality of tool descriptions affects which tool the agent picks.
  • Continuous Feedback: Writing friction notes while building produced more specific feedback than writing them from memory.

What's next for Emilia

The server is the foundation for the rest of the original vision. Planned next steps include:

  • Mobile & Wearable Expansion: A native Android app and a Wear OS companion
  • Action-Gated Workflows: Email drafting with an explicit confirmation step before anything is sent, plus presentation and projector control
  • Native Alexa+ Skill: Packaging Emilia's tools as a real Alexa+ skill so the same MCP server can serve a physical device
  • Enhanced Interview Coaching: Expanding interview practice with a larger question bank and progress tracking across sessions

Built With

Share this project:

Updates

Submission history