• 

  The full pipeline, live and deployed — from a product idea to a compliant, audited Etsy listing.

• 

  Five-agent pipeline, verified against the live code — including the compliance gate, audit retry loop, and two human-in-the-loop checkpoints

• 

  Agent 1's RAG loop — retriever_tool grounds every compliance verdict in Etsy's real policy documents, not the model's guess

• 

  Agent 4's routing fix: a rejected draft surfaces its content and requests a correction, instead of discarding human input into regeneration

• 

  Agent 5's final human approval — the last checkpoint before a listing ships, with the option to approve or send it back for revision

• 

  Live verification: a human correction was routed back correctly instead of vanishing into an automatic retry loop

Pingpin — Etsy Multi-Agent Listing System

Inspiration

I run a small Etsy shop selling enamel pins, and I write my own listing copy — in my second language. Every time a listing underperforms, I face the same unanswerable question: is it the language, or is it the product? For non-native-English sellers, this ambiguity is constant. You can write copy that's grammatically correct and still miss the tone, keyword instinct, or subtle policy nuance that a native seller would get right by default.

I also kept seeing a second, less obvious problem: Etsy's rules about what counts as "handmade," "designed by," or "curated" are genuinely easy to misunderstand, especially for sellers sourcing finished goods from wholesale markets. A listing can look perfectly fine and still get flagged after it's already live — because nobody checked the underlying selling concept against policy before writing a single word of copy.

Pingpin started from those two real, personal frictions — not from "I wanted to build an agent system and needed an excuse."

What it does

Pingpin is a 5-agent pipeline, built on LangGraph, that takes a raw product idea and turns it into a compliant, audited Etsy listing — with two points where a human explicitly stays in control:

1. Agent 1 (Compliance) checks the selling concept itself against Etsy's actual published seller policies using retrieval-augmented generation — not the model's own (often outdated) guess about the rules.
2. Agent 2 (SEO Extraction) cleans and structures market signals from user-pasted competitor listings, rejecting noise before it can pollute the copy that gets generated.
3. Agent 3 (Drafting) fuses those signals, the seller's own product concept, and tone preference into a title and description.
4. Agent 4 (Audit) runs a two-layer check: hard, deterministic Python rules first (word count, banned terms, required use-case language), then LLM soft-scoring on tone and differentiation — only after the hard gate passes. If a draft fails and automatic correction runs out, a human is asked to intervene directly.
5. Agent 5 (Delivery) formats and archives the final listing, then pauses for a human's final sign-off before anything is considered "shipped."

The whole pipeline is deployed live on Alibaba Cloud ECS behind a FastAPI wrapper, with a two-endpoint /generate → /resume API that lets a caller step through both human checkpoints over real HTTP.

This is also Condition C of a live, three-condition A/B/C experiment running on my actual Etsy shop — comparing fully human-written copy, fixed-template batch generation, and this agentic pipeline, measured against real favorites-rate and conversion data.

How I built it

• LangGraph / LangChain for orchestration — each agent is a node; Agent 1 is its own compiled sub-graph (a call_model ⇄ tool_node loop) so its retrieval-and-judge cycle stays decoupled from the rest of the pipeline.
• DashScope cloud Qwen (qwen-plus) for inference, text-embedding-v3 for Agent 1's policy-document embeddings — migrated from a fully local Ollama setup partway through development once quality on ambiguous cases became the bottleneck.
• ChromaDB as the vector store for Agent 1's Etsy policy corpus.
• Pydantic structured-output contracts everywhere an agent's output needs to be machine-checkable rather than free text — Agent 2's CompetitorSignal, Agent 4's AuditResult, Agent 1's ComplianceVerdict.
• LangGraph's interrupt() + a MemorySaver checkpointer for the two human-in-the-loop pauses, so the graph can genuinely suspend mid-run and resume later from an external HTTP call rather than blocking a terminal session.
• FastAPI wrapping the compiled graph with /generate and /resume endpoints, deployed with nohup uvicorn on an Alibaba Cloud ECS instance.

The design principle that runs through all five agents: deterministic logic belongs in Python; only genuinely subjective judgment goes to the LLM. Word counts, banned-word checks, and retry thresholds are plain if statements. Tone, differentiation, and "does this sound natural" are the only things asked of the model.

Challenges I ran into

• A model that couldn't add. Agent 4 asks the LLM to score four dimensions (0–5 each) and report a total. The self-reported total routinely didn't match the sum of its own sub-scores — once, four scores that summed to 18 came back with a reported total of 4, silently rejecting perfectly good copy. Fix: never trust the model's arithmetic. Compute the sum in Python.
• A routing bug that ate human corrections. When a human-supplied correction to a rejected draft also failed the hard gate, the system didn't tell the human — it silently discarded their edit and threw it back into the automatic AI-regeneration loop. Fixing it meant adding a last_edit_source field so the router could tell an agent-authored draft apart from a human-authored one, and re-prioritize accordingly.
• A hallucination that could have caused a real compliance problem. The pipeline once classified a standalone electronics product as an "accessory/case" for that device, pulled toward popular Etsy search terms rather than the actual product description. Fixed by promoting category to a first-class shared state field and tying Agent 1's compliance verdict to category accuracy, not just policy-text similarity.
• "It's fixed in the code" ≠ "it's fixed in production." Twice during final testing, a fix that was verified correct in the GitHub repo still didn't show up in live behavior — because the server's running process predated the fix and had simply never been restarted. Deploying a change means restarting the process, not just pulling new code onto disk.
• Dependency drift across environments. A requirements.txt generated via pip freeze on local macOS pinned exact versions (numpy==2.5.0, onnxruntime==1.27.0) that didn't exist on the deployment server's package mirror at all. Lesson: pin exact versions for reproducibility during development, but use range constraints for anything meant to be portable.

Accomplishments that I'm proud of

• The full pipeline — compliance check through final human approval — is not a local demo script. It's verified end-to-end over real HTTP requests against a live deployment, including both human-in-the-loop pause-and-resume cycles.
• The hard-gate/soft-score separation in Agent 4, and the state-as-memory design that lets a human-in-the-loop correction resume without re-running earlier agents, both held up under real adversarial testing (deliberately malformed corrections, edge-case word counts) rather than just the happy path.

What I learned

The most important thing I learned wasn't about LangGraph syntax — it was a definition. Partway through, I realized I'd been calling Agent 2 and Agent 3 "agents" just because they sat inside a LangGraph pipeline and called an LLM. They aren't, really. The test for whether something is agentic isn't "does it call a model" — it's "does the model's output actually decide where control flow goes next." Agent 2 extracts; Agent 3 drafts; neither one branches anything. The only genuinely agentic decision point in this system is the A4 judgment call (is this good enough?) combined with the deterministic policy that decides what happens next if it isn't. Everything else is a well-organized workflow, not an autonomous agent — and being honest about that distinction changed how I described (and trusted) the rest of the architecture.

What's next

The hackathon submission is Condition C of a larger experiment, not the end of it. The next phase is feeding this pipeline real Etsy sales data under actual pressure — running the full A/B/C comparison (human-written vs. fixed-template vs. this agentic pipeline) against live favorites-rate and conversion data, instead of test SKUs.

Built With

• alibaba-cloud-ecs
• chromadb
• dashscope
• fastapi
• human-in-the-loop
• langchain
• langgraph
• pydantic
• python
• qwen-plus
• rag
• uvicorn