Inspiration
B2B sales reps spend the day moving between client visits. The five minutes right after a visit — while the details are still fresh — is exactly when nobody writes anything down. The admin gets postponed, details get forgotten, follow-ups get missed. CRMs make it worse: they're forms that feel like paperwork, so reps avoid them, and the pipeline goes stale.
I wanted to test whether an AI agent could remove the form entirely: let a rep just tell the story of the visit, in their own language, and have the agent turn it into structured pipeline data — without ever inventing a fact. It also became my way of learning how an agent really works end to end: taking text and voice, extracting structured information, running a clarification conversation, keeping durable state, and firing scheduled reminders.
What it does
Send Weppy a text or a voice note after a sales visit — on Telegram (@recap_sales_bot) or in the browser app. It transcribes the voice note, then extracts four things: the institution/company, the contact person and their role, the product or service discussed, and the current deal stage. It replies with a clean pipeline card.
The key behavior: fields it isn't sure about are left empty, never invented. Weppy asks for them one at a time, each with a Skip button — so a rep who genuinely doesn't know the contact's role can skip and move on. Once the note is saved, Weppy does the follow-up work: it works out a follow-up date, sends reminders and a daily recap at the time the user chose, and exports the whole pipeline as CSV a manager can open.
Every note is private to the account that sent it. Two people can use the same bot and neither sees the other's pipeline — Telegram notes are keyed to the sender's private chat ID, web sessions to a random token in the browser. Replies are in English or Indonesian, chosen during onboarding.
How I built it
- One Cloudflare Worker (
src/worker.ts, TypeScript) serves three things: the Telegram webhook, the web JSON API (/api/session,/api/message,/api/voice,/api/export), and the static web client inpublic/— no framework, no build step. - One shared conversation engine (
src/chat.ts) drives both transports, so Telegram and the web never drift apart. - Cloudflare D1 (SQLite) stores profiles, owner-scoped notes, drafts, activity, and processed Telegram update IDs. Every query is scoped by
owner_id. - Groq does two jobs:
whisper-large-v3transcribes voice notes, and an open model (openai/gpt-oss-20b) does strict-JSON extraction of the four pipeline fields. - A Worker Cron Trigger runs every minute;
src/delivery.tsmakes an atomic D1 claim per owner / kind / date so overlapping cron runs can never send a duplicate recap or reminder. - Security: Telegram webhook requests are checked against the bot's secret token, group chats are rejected, and all secrets live in Cloudflare — never in the repository.
- Tests: 21 Vitest tests covering owner isolation, foreign note-ID callback rejection, webhook-secret rejection, web session isolation, the voice/export flows, and concurrent cron claims — all passing.
Challenges
- Not inventing data. The hardest part is making extraction honest. It's easy for an LLM to fill gaps confidently, so the guardrail lives in two places: a strict-JSON prompt that is told to leave unknown fields blank, and a UI that shows "not set" and routes missing fields into a clarification step with Skip.
- Duplicate scheduled sends. Cron triggers can overlap. I added conditional D1 date claims so each owner/kind/day is claimed at most once — a real race, with a concurrency test to prove it.
- Two transports, one brain. Pulling the conversation engine out of the Telegram handler so the web app behaves identically took refactoring, but it's what keeps the two surfaces honest.
- Running free. Workers + D1 + Groq free tiers keep it at zero cost; the trade-off is best-effort availability, which I document openly instead of pretending otherwise.
Accomplishments
- A real Indonesian voice note becomes a structured note with missing fields flagged — the "oh, this is useful" moment.
- Owner isolation that holds: a second account sees an empty pipeline, and a foreign note ID can't open anyone's data.
- One engine, two surfaces (Telegram + web) with identical behavior.
- 21 passing tests including concurrency and isolation; deployed and verified live.
What I learned
How an agent works as a system, not just a prompt: the loop of input → transcription → strict extraction → stateful clarification → durable storage → scheduled delivery. And how much of "making AI useful" is really guardrails — keeping the model from inventing, keeping users' data separate, and keeping scheduled jobs from double-firing.
What's next
- WhatsApp as a second channel (the transport layer is already abstracted).
- A shared team view for sales managers.
- Calendar sync and richer follow-up rules.
- Production-grade reliability — backups, uptime, monitoring — beyond the free demo tier.
Built With
- ai-agent
- cloudflare-d1
- cloudflare-workers
- cron
- css
- groq
- html
- javascript
- json
- llm
- markdown
- rest-api
- speech-to-text
- sqlite
- telegram-bot-api
- typescript
- vitest
- voice
- web-app
- whisper-large-v3
- wrangler
Log in or sign up for Devpost to join the conversation.