Feedloop

Inspiration

A refund is feedback with a price tag.

I run Nulnul Tour, a small travel site for Seoul. It's just me. Visitors already told me what was wrong: a coupon button that didn't work, a question about a café's opening hours. Every time, I typed the same reply: "Thanks, we'll fix it next time." And "next time" often never came.

So I built Feedloop. It answers visitors, groups the same complaints, and opens a pull request that fixes the cause. By September it had handled 22 real visitor messages on Nulnul Tour, and three other people have signed up and connected it to two services.

Now I'm planning to sell paid tour passes with PayPal. Selling brings a new kind of message: "Where's my order?" "Can I get a refund?" A solo operator can't answer those at 2 a.m. — and a customer at 2 a.m. would rather get a clear answer and a refund right away than a human reply two days later. But handing your money to an AI is scary.

And a refund is not just a cost. Every refund has a reason — a broken coupon, a confusing plan page, a missing feature. If the reason disappears into a refund log, the next customer hits the same wall.

So Feedloop learned two things during this hackathon: to settle payments safely, and to turn every refund reason into the next thing to fix.

Why it matters. Refunds are a constant, not an edge case: an estimated 19.3% of online sales were expected to be returned in 2025 (NRF & Happy Returns, 2025). When a refund request goes unanswered, the customer can go to their bank instead. Mastercard expects 324 million chargebacks by 2028, up 24% (Mastercard, 2025), costing merchants a forecast $42 billion (Mastercard press release, 2025). Big retailers have support teams for this. A solo operator has a chat box and no time — and no one reading the reasons behind the refunds.

What it does

Add one line to your site. Feedloop answers your customers' everyday questions, sells, and settles refunds with PayPal inside limits you set, then turns what they said into fixes.

1. It settles the money — safely.

• A customer asks for a refund in chat and gives their PayPal order ID and the email they paid with. Both must match before the agent talks about the payment.
• The server reads the capture from PayPal (amount, currency, date) and checks the limits. The model only passes on the customer's reason (and an amount, for a partial refund).
• Inside the owner's limits (default: up to $20, within 14 days, $100/day, one refund per customer per 30 days) the agent refunds on its own and shows a PayPal card: "Refund of $9 sent to PayPal."
• Anything bigger becomes a one-tap approval card in the owner's inbox, showing the amount, the PayPal capture, and which limit was exceeded.
• The card also shows how the owner decided similar requests before — e.g. "Similar amounts: 4 of 5 refunded" — with links to those past conversations. The suggestion is a plain rule over the owner's own decisions, not a model guess, and the owner still taps Approve or Reject.
• It can also sell: the agent offers a plan from the price table and shows a Pay with PayPal card mid-conversation.

2. It turns refund reasons into work. The refund reason goes through the same AI classifier as all other feedback. Each group of similar reports also gets a one-line verdict — Fix, Won't fix, or Owner decides — so the owner sees at a glance what is a bug and what is a product decision. The verdict is a label only; nothing happens until the owner acts.

• Bug / UX → similar reports are grouped, and Feedloop can open one pull request that fixes the cause. A human decides whether to merge.
• Feature request → grouped, and the owner can turn the group into a spec draft with one click.
• Everything else (price, expectations) → counted, not lost.

The dashboard's Refund reasons card shows every refund request by kind (Bug / UX, Feature request, Other, Not classified yet) and the top 3 reasons, each linked to its fix PR or conversation. A reason that suddenly jumps (3 or more in the last 24 hours and at least twice its usual daily rate) gets a Spike badge — often the first sign that a release broke checkout.

3. The owner sees everything in one place.

• Dashboard: payment inquiries, auto refunds, refunds waiting for approval, and Refund reasons.
• Payment ledger built with AG Grid: every purchase and refund in one table, filtered by type and status, one click from the conversation.
• Inbox: each payment conversation has a Payment activity panel.
• Refund Studio, built with AG Studio: a dashboard on a labeled sample store with a custom agent, Refund Watch. Ask whether a refund reason is repeating, and it compares time windows, finds the fix for that reason, and places an alert at the top of the dashboard.
• Per-bot PayPal settings: paste your sandbox keys, the webhook registers itself, set limits, and flip Pause automatic refunds or Ask me before every refund.

What the video shows. The example store is scripted so the story is easy to follow: Nulnul Tour is a real site, but paid passes are planned, its coupon bug is staged, and its chat replies are scripted. The approval refund is a real PayPal sandbox run. Refund Studio is a real production run on labeled sample store data, sped up in the video. Owner screens marked "Sample data" use sample data. The fix diff is a real pull request Feedloop wrote against a demo copy of the site. Music: "Inspired" by Kevin MacLeod (incompetech.com), CC BY 4.0.

How we built it

• Next.js 16 (App Router) on Vercel, Supabase (Postgres), NextAuth (Google sign-in), next-intl (English first, EN/KO).
• Claude (Anthropic) through Vercel AI SDK 7 runs the chat agent and the feedback classifier. Gemini or OpenAI can be selected instead.
• PayPal REST API (sandbox): Orders create/read/capture, Captures read/refund, Refunds read, Disputes list (guardrail check), Webhooks register/list/update/delete, and webhook signature verification done locally.
• PayPal Agent Toolkit for read-only lookups (get_order, get_refund) through our AI SDK 7 adapter — behind a server guard: nothing is read until a payment is verified in this conversation, and results are cut to a whitelist of fields.
• Money-moving calls never go through the LLM. Capture and refund run on our own server tools and REST client, with a PayPal-Request-Id on every call so a retry can't refund twice.
• One Postgres function decides and reserves a refund (payment_action_open_refund) before PayPal is called, in one transaction — two "refund me" messages at the same time can never both execute.
• AG Grid Community for the payment ledger.
• AG Studio (Studio Agent Framework) for Refund Studio: a custom agent with three tools, a custom Reason → Fix widget, and a Feedloop theme, running on Claude through our own LLM adapter and a server route.
• Built with Claude Code as the coding agent.

Challenges we ran into

• Letting an AI move money without trusting the AI. We moved every money decision to the server: the paid amount and date come from PayPal's capture, limits live in one database function, and the sentences that announce a refund are fixed server text, not model output.
• Prompt injection vs. toolkit write tools. The Agent Toolkit's write tools would let a visitor talk the model into any amount. We block them twice (allowlist + name filter) and use the toolkit only for guarded reads.
• Webhooks that arrive late or twice. Orders whose buyer closed the tab are captured from CHECKOUT.ORDER.APPROVED, and refunds or reversals made outside Feedloop are synced back.
• Judges testing a real guardrail. The "one auto refund per customer per 30 days" rule also applies to judges reusing a sandbox buyer. We give three buyers and made the other path part of the demo: a repeat refund becomes an approval card the judge can approve.

Accomplishments that we're proud of

• Real sandbox purchases and refunds end-to-end — one automatic, one approved from the card.
• A 22-case refund guardrail scenario that runs against a throw-away Postgres (npm run db:dry-run).
• A refund reason can travel all the way from a chat message to a pull request that fixes the cause.

What we learned

• The hard part of "agentic commerce" is not calling the payment API. It's deciding what the agent is never allowed to decide. Anything that moves money and can't be undone goes through a few small, hard-coded gates on the server; the model never has the last word.
• The loop has to close where it started. When a fix PR is merged, the customer who reported the problem hears "fixed" in the same chat — not in a changelog they will never read.
• Refunds are feedback with a price tag. Small operators already have the data to fix the cause — it's just sitting in the wrong place.

What's next for Feedloop

• Use it on Nulnul Tour when paid tour passes launch, and measure whether repeat refund reasons go down. (Goal — not measured yet.)
• Act on disputes (today CUSTOMER.DISPUTE.* webhooks are stored only).
• A 24-hour reminder for approval cards that nobody has opened.
• More payment providers behind the same guardrail.

Built With

• ag-charts
• ag-grid
• ag-studio
• anthropic
• claude
• claude-code
• next-intl
• next.js
• nextauth
• paypal
• paypal-agent-toolkit
• paypal-rest-api
• postgresql
• react
• supabase
• typescript
• vercel
• vercel-ai-sdk