Power

Helps independent hawkers automate the repetitive work — reservations, WhatsApp enquiries, reviews, and more — so you can focus on the food.

Comment 1

Inspiration

Most restaurant owners manage WhatsApp queries manually — reading messages, replying one by one, forgetting who's a regular. We built Arpo to handle that automatically.

But the deeper inspiration came from watching AI agents fail in the same ways repeatedly. A chatbot that forgets to ask for party size. One that gives wrong hours. One that's too stiff with regulars. Traditional bots get deployed and stay broken until a developer manually patches them.

We wanted to build something different: an agent that watches itself fail, understands why, and fixes its own behaviour before the next shift.

What it does

Arpo is a WhatsApp AI front desk for Singapore restaurants. Customers book tables, ask about the menu, and get answers in English, Mandarin, and Singlish — instantly, 24/7.

What makes it different is the self-improvement loop:

  1. Every conversation is traced to Arize Phoenix via OpenInference instrumentation
  2. After each conversation, an LLM-as-judge evaluator scores it on four dimensions: task completion, accuracy, tone, and efficiency (1–5 each)
  3. If the composite score falls below 4.5, the self-improvement loop fires automatically
  4. The agent reads its own failed conversation from Phoenix via MCP, reads its current system prompt, identifies the failure pattern, and rewrites the prompt to fix it
  5. The new prompt version is saved to Phoenix via upsert-prompt — fully versioned and auditable
  6. The next conversation loads the improved prompt from Phoenix via get-latest-prompt

The system prompt evolves autonomously over time. Every failure makes Arpo slightly better at its job.

Owners can also text Arpo directly to query operational insights — average eval scores, what customers are asking about, recent conversation quality — all answered by an Analytics Agent that queries Phoenix MCP in real time.

How we built it

Gemini 2.5 Flash via the Vertex AI SDK (@google-cloud/vertexai) is the core LLM. We built a thin adapter layer that wraps Gemini in a unified LLMClient interface so all three agents — Customer Agent, Eval Agent, and Self-Improvement Agent — share the same interface.

Arize Phoenix does the heavy lifting on observability. Every Gemini call is automatically traced via @arizeai/phoenix-otel. The Phoenix MCP server (@arizeai/phoenix-mcp) runs as a child process and gives the agent programmatic access to its own traces — list-traces, get-spans, get-latest-prompt, upsert-prompt.

Meta WhatsApp Cloud API is the customer-facing channel. Messages arrive at a Next.js webhook (/api/whatsapp-meta), which routes to either the Customer Agent (any number) or the Analytics Agent (owner's number).

Neon Postgres + Upstash Redis handle persistent state: customer profiles, booking records, and per-conversation history.

Restaurant onboarding is fully automated: the owner pastes their website URL, Gemini extracts structured menu data, and the agent is ready to answer questions about their specific restaurant in under 30 seconds.

Challenges we ran into

MCP in serverless. The Phoenix MCP server runs as a stdio child process. Vercel's serverless functions prune packages that aren't statically imported, so the binary was silently missing at runtime. Fixed with outputFileTracingIncludes in next.config.mjs — but only after several baffling "module not found" errors that pointed nowhere obvious.

Serverless function lifetime kills fire-and-forget. We initially ran the eval and self-improvement loop as async background work after sending the WhatsApp reply. Vercel terminates the function as soon as the response is sent — meaning the eval silently never ran. Fixed by awaiting both the eval and the self-improvement loop before returning.

Self-improvement truncating the prompt. With max_tokens: 2000, the rewrite agent would return a partial prompt — improving the first half and cutting off mid-sentence. Raised to max_tokens: 4000 and added explicit instructions to return the complete prompt with all existing sections intact.

Phoenix prompt name normalisation. Phoenix strips hyphens from prompt identifiers — arpo-system-prompt becomes arposystemprompt. Took a few confused "prompt not found" errors before we discovered this and standardised on the stripped form throughout the codebase.

Accomplishments that we're proud of

The self-improvement loop works end-to-end in production. A real WhatsApp conversation ends, the eval fires, a low score triggers the rewrite, and the next conversation loads a measurably better system prompt — all without any human intervention. Watching it happen live for the first time was genuinely surprising.

We also built a fully auditable prompt history. Every version of Arpo's system prompt is versioned in Phoenix with a timestamp. You can see exactly how the agent's behaviour evolved over a day of conversations and roll back to any prior version instantly.

What we learned

Observability is load-bearing for self-improving agents. Without Phoenix tracing every span, there's nothing for the agent to read about itself. The MCP layer that lets the agent query its own traces is what closes the feedback loop — it's not an optional add-on, it's the architecture.

LLM-as-judge evals need to be specific. A vague "rate this conversation 1–10" gives noisy, inconsistent scores. Breaking it into four concrete dimensions (task completion, accuracy, tone, efficiency) with explicit 1–5 rubrics produces scores stable enough to actually trigger meaningful improvements.

Serverless and long-running agent loops don't mix without careful design. Vercel's execution model assumes fast, stateless functions. Any agentic work that needs to complete after the HTTP response — eval, self-improvement, trace flushing — must be explicitly awaited or moved to a queue.

What's next for Arpo AI

Owner dashboard. A real-time web view of bookings, customer profiles, and eval score trends — so owners can see what Arpo is doing without texting it questions.

Automated booking reminders. A cron job sends confirmation 24 hours before each reservation — reducing no-shows, which is the number-one complaint from Singapore restaurant owners.

Multi-restaurant support. Right now Arpo is hardcoded to one restaurant. The next step is a proper onboarding flow where any owner can sign up, connect their WhatsApp number, and get their own isolated agent instance.

Built With

Share this project:

Updates