Echo

Show Echo any workflow once. It writes the SKILL.md, parameterizes the values, and ships a portable skill any AI agent can run generalized, human-approved, no lock-in.

Comment

Inspiration

Every team pays the same hidden tax. The same form filled out 50 times a week. The same Monday Slack post. The same five links checked every morning.

Today's options are all bad. Code it yourself — and most people can't. Use a no-code bot — and it breaks the first time a website changes a button. Fine-tune an LLM — and you've started a research project.

I wanted something simpler: show the AI the work once, and have it generalize from the one example. I had read open-source Skill Recorder watches your screen and ships a portable SKILL.md. It had the right shape, but it was a desktop app, it required GitHub Copilot CLI, and it didn't actually generalize. I wanted the same pipeline on the web, against any service, with a human in the loop at every step.

What it does

Echo is a platform with two halves.

1. The Recorder. A browser page at /record that captures your screen, records your voice if you want it, and ships the whole thing to an AI. The AI returns a structured analysis — the title, the intent, the ordered steps. You review and approve it. Then a second AI turns your steps into a generalized plan — your hardcoded search query becomes a {{job_query}} token, your "for the 3 rows" becomes "for every row", and every step is mapped to a tool Echo knows about. You review and approve that. The final output is a portable SKILL.md file that any AI agent runtime can read and run.

2. The Composer. A 2×2 grid at /compose. Each slot owns its own goal, plan, run, and progress stream. Hit dispatch on all four and they fire in parallel — real headless browser, real screenshots, real action logs streaming back to your screen. The runs page shows a "📸 real headless" badge whenever the action came from a real browser rather than a simulation.

How we built it

  • Next.js 16 on Vercel with the function timeout pushed to 60s so the LLM calls have breathing room. State is local-first (everything in your browser's localStorage), with an optional Firestore mirror when GCP is configured.
  • Real headless Chromium running inside the Vercel function. Not a fake — every screenshot in the runs page is from a live browser session. The browser-executor/ package is a Cloud Run-ready container for production scale.
  • Gemini 3.5 Flash for both the describer and the builder. The code tries Vertex AI first, falls back to AI Studio, and has a fallback chain of three models.
  • A 4-stage pipeline lifted from Skill Recorder (Record → Timeline → Describer → Builder → SKILL.md), adapted from Electron to a pure web app. The describer and builder prompts are ~250 lines each, with strict schema validators. The {{id}} token system is a port of theirs.
  • An eval harness that runs 6 fixture recordings against the real LLM and scores each one on tokenization, tool picks, step count, and side-effect classification. Average score: 64% (above the 60% gate). The harness caught a real prompt regression the first time I ran it.
  • Human-in-the-loop gates between every LLM step. If a step is wrong, you say so in plain English ("step 3 is irrelevant") and the agent re-runs with your feedback in context.

Challenges we ran into

  1. Vercel's 10-second function timeout. I pushed it to 60s, warm-started the browser, and split the pipeline into separate routes so each LLM call is its own function. A real production deploy would use Cloud Run with no timeout pressure.
  2. Chromium binary stripping on Vercel. The browser's bin/ directory kept getting dropped. Fixed with serverExternalPackages and outputFileTracingIncludes so the binary ships inside the function bundle.
  3. Cloud Run worker failed on first deploy because the org blocks JSON service-account keys. I switched to IAM-bound credentials and ran out of time to retry, so the live demo runs on Vercel-hosted Chromium. The Cloud Run Dockerfile is ready when those constraints relax.
  4. The LLM kept hardcoding values instead of using the {{token}} pattern. The first eval run scored 30% on a fixture because Gemini wrote "AI jobs" instead of {{job_query}}. I rewrote the Builder prompt with an explicit "Dynamic values are tokens. Period." section. The score went to 60%+ but the LLM still misses abstract-noun tokens about half the time — a self-critique pass is the next iteration.
  5. A React hooks rule violation on the agent detail page caused a "This page couldn't load" error. Discovered during a smoke test right before the demo. Fixed in two lines.

Accomplishments that we're proud of

  • End-to-end pipeline live at echo-one-liard.vercel.app. Record → analyze → build → SKILL.md, with human review gates between every step.
  • Real headless integration, not a mock. Every screenshot in the runs page is from a real browser session.
  • Multi-composer parallelism. Four agents fire in parallel; each one streams its own progress, browser actions, and screenshots.
  • Portable SKILL.md export. Every saved skill is a self-contained markdown file with YAML frontmatter, parameterized body, rules, and error codes. Drop it into any agent runtime.
  • The eval harness caught a real gap before users did. The tokenization regression was visible in the scorecard the moment I changed the prompt.
  • Local-first architecture. The app degrades gracefully without GCP — everything works on localStorage alone.

What we learned

  • The 4-stage pipeline is the right shape. Every alternative I considered — single-shot LLM, fine-tuned model, end-to-end agent — lost the human-in-the-loop gates, and those gates are what kept hallucinations from shipping as skills.
  • The hard part isn't the LLM call, it's the data model. Tokens, plan structure, step kinds, tool mapping, allowed-tools — none of these are free from the LLM. Each one needs a strict schema with hand-rolled validation.
  • Headless browsers on Vercel are real but bounded. 60s is the absolute ceiling. For real production scale, the worker belongs on Cloud Run or Fly.io, not on the same Vercel function as the consumer app.
  • Real-time eval is the only way to know if a prompt change helped. I lost hours to LLM variance on the first run. A 6-fixture suite is the floor; 20+ is what you need for statistical confidence.
  • Browser-level event capture is enough for 80% of use cases. Skill Recorder captures OS-level events via Electron. We can't do that in a web app. But for browser-first workflows (LinkedIn, Gmail, HubSpot, Sheets, Notion), the browser API gives us everything we need.

What's next for Echo

  1. A self-critique pass in the Builder — a second LLM call that asks "did you miss any tokens?" before finalizing. Should push the eval from 64% to 80%+.
  2. Production Cloud Run deployment of the headless browser worker once the org's JSON-key constraint is worked around.
  3. Real OAuth integrations with HubSpot, Notion, Slack, and Gmail. The Builder can already pick these tools; what's missing is the OAuth handshake.
  4. A Chrome extension for OS-level event capture
  5. Scheduled runs / automations — a Skill Recorder "AutomationBuilder" tier that wraps a plan in a trigger (schedule, webhook, condition).
  6. A 20+ fixture eval suite + a regression test for the Builder prompt.
  7. A token marketplace — a public library of {{value}} tokens users have already configured, so skills are shareable across teams.

Echo is open-source at github.com/Prasannaverse13/Echo. Live demo at echo-one-liard.vercel.app.

Built With

Share this project:

Updates

Submission history