E-Commerce Dispute Investigator

We don't automate refunds. We automate the investigation before the refund.

Comment

Inspiration

Ecommerce disputes look like a support problem. For the business, they are a money problem and a time problem.

Here is a simple example. A customer says an $850 laptop never arrived and asks for a full refund. If the team refunds without checking tracking, delivery photos, and policy, that is $850 lost in one click. If a careful person investigates by hand, it can still take about 25 to 40 minutes, because the facts sit in different places: orders, payments, shipping, warehouse, and policy docs.

We can write the time cost like this:

$$ \text{Monthly investigation cost} = N \times T \times W $$

where (N) is messy disputes per month, (T) is hours per manual investigation, and (W) is the loaded hourly wage.

Example: (N = 200), (T = 0.5) hours, (W = \$40):

$$ 200 \times 0.5 \times 40 = \$4{,}000 \text{ per month} $$

Refund leakage on high-value misses looks like this:

$$ \text{Monthly leakage risk} = K \times \$850 $$

If that kind of miss happens 10 times in a month:

$$ 10 \times 850 = \$8{,}500 $$

Now compare that with the agent. One investigation is a short tool-calling run on a small model, plus a cheap embedding search for policy, plus an optional voice brief. In rough terms:

$$ \text{Agent cost per case} \approx C_{\text{LLM}} + C_{\text{embed}} + C_{\text{voice}} $$

For our stack, that is usually well under \$0.05 per case for the text investigation, and still cents even with a short ElevenLabs brief. So the shape is:

$$ \text{Human cost per case} \gg \text{Agent cost per case} $$

Example: 30 minutes at \$40/hour is about \$20 of human time. The agent draft is closer to pennies. The human still reviews and decides. We save time and reduce bad refunds, without giving the model the wallet.

We do not claim these are exact customer numbers. They show the shape of the pain. Our live demo then proves the point on a real \$850 case.

We saw too many tools that either chat nicely or refund too fast. We wanted something else: an agent that investigates first, then helps a human decide. Our idea stayed clear from day one.

We don’t automate refunds. We automate the investigation before the refund.

What it does

E-Commerce Dispute Investigator is an operator desk for dispute cases.

A manager opens a case and clicks Investigate. An AI agent starts a short tool-calling loop. It does not call every tool every time. It picks the tools that fit the claim. A duplicate charge case needs payments. A wrong item case needs the warehouse pick. A “not received” case needs tracking and delivery evidence.

This is important: the product is not a RAG chatbot. Order facts, GPS, photos, and payments come from tools and the database. RAG is only for policy. When the agent needs company rules, it searches policy documents with embeddings. Policy helps explain the recommendation. It does not invent shipping facts.

The desk also streams the tool trail live, so a manager can see each tool call as it runs.

Then the desk shows a structured Finding: evidence, contradictions, policy citations, risk, and a recommendation. Brief me turns that Finding into a short ElevenLabs voice brief, so a manager can listen in under a minute. The human still clicks approve, reject, escalate, or request info. That decision goes into the audit log. The AI suggestion is not a payment.

Our main money example is case 1042: an \$850 “not received” claim. Strong delivery evidence leads to HOLD, not a blind refund. Case 1087 shows a fair refund path for a true duplicate charge. Case 1112 asks for more info when the warehouse matches the order but the customer claims the wrong item.

How we built it

We built two apps. The frontend is a Next.js desk. The backend is an Express API.

The agent uses native tool calling in a bounded loop, so it cannot run forever. That also keeps cost low, because the model cannot spin endless rounds. Tools go through a safe gateway. The model never runs raw SQL. Findings must match a shared schema, so the UI and API stay aligned. Investigate progress is streamed live with server-sent events, so the desk can show each tool call as it happens.

For knowledge, we store policy documents in Postgres with vector search. The agent can call policy search when it needs rules. For voice, we use ElevenLabs after the investigation, as a real manager step, not a side demo. We also built a small eval set so we can check recommendations against expected outcomes.

We hosted the desk on Vercel and the API on Railway, with Postgres for both ops data and policy embeddings. We chose a small chat model and small embeddings on purpose, so each case stays cheap while still being useful.

Challenges we ran into

Cloud reliability hurt us more than we expected. When the database was cold or suspended, tools failed in random ways. One run a delivery tool timed out. Another run policy search failed. The agent often recovered in a second round, but it made live demos stressful.

The model also made soft mistakes. On wrong-item cases, it sometimes wanted to reject too fast, even when warehouse data matched the order and policy said “request a photo first.” Prompts helped, but we still needed a small hard rule for that case so the live demo stayed trustworthy.

Eval taught us another hard lesson. If the answer key is wrong, a good investigation looks like a fail. We fixed labels so the score matched real evidence, not wishful refunds.

Voice keys and quotas also failed at bad times. That forced us to treat Brief me as part of the launch checklist.

Accomplishments that we're proud of

We shipped a working investigation desk, not just slides. The full path works live: Investigate, live tool trail, Finding, voice brief, human decision, audit.

We protected a clear money story. On the \$850 case, the system holds instead of paying blindly:

$$ \text{Blocked bad refund} = \$850 $$

And we kept the unit economics honest:

$$ \text{Agent draft cost} \ll \text{Human investigation cost} $$

We also showed that fair refunds still exist when payments prove a duplicate charge.

We kept the design honest. Tools handle facts. RAG handles policy. Humans handle money. That split feels like a real ops product.

What we learned

A useful ops agent needs more than a good prompt. It needs tools for facts, light RAG for policy, clear output structure, eval for honesty, and a human gate for money. It also needs a cost story: cheap draft investigation, expensive human time saved, and bad refunds blocked.

We also learned that business judges need the story fast. Show the money risk. Show the time save. Show the agent cost versus human cost. Show that the agent investigates and the human still decides. Deep tech matters more when that story is already clear.

What's next for E-Commerce Dispute Investigator

Next we want real store and carrier connections instead of demo data. We want stronger login roles, longer audit history, and a manager queue with deadlines.

On the AI side, we want eval in CI, better policy search checks, token-cost dashboards per case, and safer support for more than one merchant. The long-term goal is simple: become the investigation layer in front of refunds, while humans keep final control of the money.

Built With

Share this project:

Updates

Submission history