Isse Automate Kardo

Meet Hexi, an Alexa+ MCP assistant that finds where your time goes, pinpoints the bottleneck step and says "Isse Automate Kardo." It remembers what you dismiss, so it never nags twice.

Comment

Inspiration

Most of the time people lose at work is not in big tasks. It is in the same six-step routine repeated every week: pull the numbers, paste them, format, send. Nobody can say which step is the slow one, and nobody can say whether automating it would pay back. Existing tools either log your time or build automations. None of them tells you what is worth automating, how many hours it would save, and how sure it is.

I wanted that to be a question you can ask out loud. "Isse Automate Kardo" ("automate this for me") became Hexi, an Alexa+ assistant that finds the repetitive workflows in your activity, explains the bottleneck, and proposes automation only after you say yes.

What it does

Hexi is a self-hosted MCP server (Streamable HTTP, spec 2025-11-25) with 16 tools, plus an Agent Skill (skills/friction-radar/SKILL.md) that tells an agent which tool to call and which rules to follow.

  • Discovers workflows from raw events with no labels (discover_workflows)
  • Ranks friction by total time cost (get_top_friction_points)
  • Scans everything for automation and ranks by hours saved per week, explaining every refusal (find_automation_opportunities)
  • Debugs a workflow step by step and flags unusual runs
  • Diagnoses: findings, prescriptions, hours saved with an interval, and a 0-100 reliability grade (diagnose_workflow)
  • Propose, then confirm: propose_automation never executes. confirm_automation runs only after an explicit yes, once, within an hour.
  • Remembers dismissals ("stop suggesting bug triage") across restarts.

It works on data it has never seen: pasted step lists, Toggl CSV, ActivityWatch JSON, a public GitHub repository's pull-request history, or a link-shared Google Sheet. The browser demo (retro, pixel-style) has a hexagonal mascot whose expression follows what the analysis is doing, with text chat and a voice loop that simulates "Alexa, ask Hexi...".

How we built it

Two doors lead to one core. Alexa+ or any MCP client talks to src/mcp_server.py over Streamable HTTP. The web demo goes through web/mcp_bridge.py, a real MCP client that refuses servers older than 2025-11-25, and falls back to in-process calls if the server is down.

The analysis is deliberately inspectable rather than an LLM's guess:

  • Workflow discovery adapts process mining (van der Aalst et al., 2004; Bose & van der Aalst, 2009). Runs are clustered by normalised longest common subsequence, which tolerates reordered, extra or missing steps:

$$\mathrm{sim}(a,b)=\frac{\mathrm{LCS}(a,b)}{\max(|a|,\,|b|)}$$

  • Unusual runs use Tukey's fence, flagging a duration $d$ when $d > Q_3 + 1.5\,(Q_3 - Q_1)$. Run durations are right-skewed, so this beats a z-score.
  • Automation safety scores each step from its verb and its consistency. Steps that appear in fewer than 60% of runs, need rework in more than 20%, or need judgment are refused, with the reason.
  • Time saved replays the proposal against your own history in shadow mode, keeping 10% of each step as supervision time. The estimate carries an 80% bootstrap interval (Efron, 1979) and is checked on a chronological holdout.
  • Reliability is a transparent weighted score over six components: evidence, stability, coverage, cohesion, holdout accuracy, outcome data.
  • Words come from templates over computed numbers. An optional LLM may rephrase the diagnosis, and its reply is discarded if it contains a figure the report does not support.

Stack: Python 3.12, FastMCP (official MCP SDK), Flask, NumPy, scikit-learn, plain JavaScript with the Web Speech API, Docker Compose, pytest and GitHub Actions CI. The Open Source mini challenge is covered by PR #122 to ActivityWatch/aw-client.

Challenges we ran into

  • No labels, no workflows. Cutting a raw event stream into runs using only timing recovers true run boundaries with an Adjusted Rand Index of 0.905 on my evaluation data. The errors are systematic: rare workflows get merged with their neighbours. I documented this instead of hiding it.
  • Making "automate it" safe. A voice assistant that acts on a misheard command is dangerous. Propose and confirm are separate tools, confirmation must come in a later turn, proposals expire, and nothing can be confirmed twice.
  • Safe import of untrusted data. Imports only parse and never execute. File imports are restricted by directory, extension and size. Every imported string is escaped in the page, and the server sends a strict CSP and rate-limits.
  • Stale servers. An old process holding the port made fixes look broken, so the page now shows the build id in its title bar.
  • No Alexa+ surface for developers. I could not test inside real Alexa+, so the voice loop is simulated in the browser and says so.

Accomplishments that we're proud of

  • A genuine MCP server and Agent Skill on the 2025-11-25 spec, exercised by protocol tests, not just mentioned in the README.
  • Honest numbers: every saving comes with an interval and a reliability grade, and weak data cannot earn an A.
  • 190+ tests, CI on every push, and a one-command Docker deployment.
  • An upstream contribution to ActivityWatch during the hackathon window.

What we learned

  • Consent design matters more than model quality in a voice-first agent.
  • A heuristic you can explain beats a model you can't, especially for advice that tells people what to automate.
  • Validating an LLM's output against computed facts is cheap and removes the worst failure: confident invented numbers.

What's next for Isse Automate Kardo

  • Wire confirm_action to a real executor (it already sits behind the consent check).
  • Test the Google Sheet import against live sheets, and Hexi on a real Alexa+ surface.
  • Learn step safety from labelled outcomes instead of verb lists, and support non-English step names.
  • Handle interleaved workflows in segmentation.

Built With

Share this project:

Updates

Submission history