Inspiration

B2B sales reps spend the day moving between client visits. The five minutes right after a visit — while the details are still fresh — is exactly when nobody writes anything down. The admin gets postponed, details get forgotten, follow-ups get missed. CRMs make it worse: they're forms that feel like paperwork, so reps avoid them, and the pipeline goes stale.

I wanted to test whether an AI agent could remove the form entirely: let a rep just tell the story of the visit, in their own language, and have the agent turn it into structured pipeline data — without ever inventing a fact. It also became my way of learning how an agent really works end to end: taking text and voice, extracting structured information, running a clarification conversation, keeping durable state, and firing scheduled reminders.

What it does

Send Weppy a text or a voice note after a sales visit — on Telegram (@recap_sales_bot) or in the browser app. It transcribes the voice note, then extracts four things: the institution/company, the contact person and their role, the product or service discussed, and the current deal stage. It replies with a clean pipeline card.

The key behavior: fields it isn't sure about are left empty, never invented. Weppy asks for them one at a time, each with a Skip button — so a rep who genuinely doesn't know the contact's role can skip and move on. Once the note is saved, Weppy does the follow-up work: it works out a follow-up date, sends reminders and a daily recap at the time the user chose, and exports the whole pipeline as CSV a manager can open.

Every note is private to the account that sent it. Two people can use the same bot and neither sees the other's pipeline — Telegram notes are keyed to the sender's private chat ID, web sessions to a random token in the browser. Replies are in English or Indonesian, chosen during onboarding.

How I built it

  • One Cloudflare Worker (src/worker.ts, TypeScript) serves three things: the Telegram webhook, the web JSON API (/api/session, /api/message, /api/voice, /api/export), and the static web client in public/ — no framework, no build step.
  • One shared conversation engine (src/chat.ts) drives both transports, so Telegram and the web never drift apart.
  • Cloudflare D1 (SQLite) stores profiles, owner-scoped notes, drafts, activity, and processed Telegram update IDs. Every query is scoped by owner_id.
  • Groq does two jobs: whisper-large-v3 transcribes voice notes, and an open model (openai/gpt-oss-20b) does strict-JSON extraction of the four pipeline fields.
  • A Worker Cron Trigger runs every minute; src/delivery.ts makes an atomic D1 claim per owner / kind / date so overlapping cron runs can never send a duplicate recap or reminder.
  • Security: Telegram webhook requests are checked against the bot's secret token, group chats are rejected, and all secrets live in Cloudflare — never in the repository.
  • Tests: 21 Vitest tests covering owner isolation, foreign note-ID callback rejection, webhook-secret rejection, web session isolation, the voice/export flows, and concurrent cron claims — all passing.

Challenges

  • Not inventing data. The hardest part is making extraction honest. It's easy for an LLM to fill gaps confidently, so the guardrail lives in two places: a strict-JSON prompt that is told to leave unknown fields blank, and a UI that shows "not set" and routes missing fields into a clarification step with Skip.
  • Duplicate scheduled sends. Cron triggers can overlap. I added conditional D1 date claims so each owner/kind/day is claimed at most once — a real race, with a concurrency test to prove it.
  • Two transports, one brain. Pulling the conversation engine out of the Telegram handler so the web app behaves identically took refactoring, but it's what keeps the two surfaces honest.
  • Running free. Workers + D1 + Groq free tiers keep it at zero cost; the trade-off is best-effort availability, which I document openly instead of pretending otherwise.

Accomplishments

  • A real Indonesian voice note becomes a structured note with missing fields flagged — the "oh, this is useful" moment.
  • Owner isolation that holds: a second account sees an empty pipeline, and a foreign note ID can't open anyone's data.
  • One engine, two surfaces (Telegram + web) with identical behavior.
  • 21 passing tests including concurrency and isolation; deployed and verified live.

What I learned

How an agent works as a system, not just a prompt: the loop of input → transcription → strict extraction → stateful clarification → durable storage → scheduled delivery. And how much of "making AI useful" is really guardrails — keeping the model from inventing, keeping users' data separate, and keeping scheduled jobs from double-firing.

What's next

  • WhatsApp as a second channel (the transport layer is already abstracted).
  • A shared team view for sales managers.
  • Calendar sync and richer follow-up rules.
  • Production-grade reliability — backups, uptime, monitoring — beyond the free demo tier.

Built With

Share this project:

Updates

Submission history