Inspiration
Every day, businesses make decisions based on stale data. A logistics company wants to know which restaurant kitchens cause the most delivery delays. A hedge fund needs real-time foot traffic patterns before they publish. A city planner needs to understand where cyclists actually feel unsafe — not where the official reports say they do.
They ask ChatGPT. ChatGPT gives them a confident, well-formatted answer drawn from data that is months or years old, with no first-hand grounding.
The insight behind UNMADE is simple: there is an entire category of questions that AI fundamentally cannot answer — not because the models aren't smart enough, but because the answers don't exist in any training corpus. They exist only in the lived experience of people on the ground, right now.
We wanted to build the missing layer: a marketplace that routes those unanswerable questions to the humans who can answer them, and pays them instantly.
What it does
UNMADE is an AI-native intelligence marketplace with three actors:
Requesters (businesses) describe a question they can't answer with existing tools. An AI agent decomposes that question into discrete field tasks, writes validation criteria, and estimates a reward per accepted observation.
Contributors (anyone with a smartphone) browse open tasks, submit first-hand observations, and receive PayPal payouts the moment their submission is accepted.
The AI layer sits in the middle: validating submissions in real-time, detecting off-topic or second-hand responses, and — once enough observations are collected — fusing them into a structured intelligence product.
How we built it
Layer Stack Backend FastAPI + SQLAlchemy async + SQLite AI Validation Claude Haiku (real-time pre-validation) + Claude (final submission gate) Payments PayPal Payouts API (instant payout on acceptance) Frontend Next.js 14 App Router + Tailwind CSS
Agent Decomposition — When a requester describes their question, Claude Sonnet breaks it down into discrete field tasks, writes per-task validation criteria, and suggests reward amounts based on the complexity of each observation required.
Real-time Pre-validation (Claude Haiku) — As a contributor types, Haiku scores the draft across four dimensions: relevance to the task, specificity (names, times, numbers), first-hand voice, and completeness. The contributor sees live feedback before they submit — so rejections become rare.
Final Submission Gate (Claude Sonnet) — On submit, Sonnet makes a binding accept/reject decision. Accepted submissions trigger an immediate PayPal payout. Rejected ones return a specific reason so the contributor can improve and resubmit.
Intelligence Fusion — Once all submission slots are filled, Sonnet reads every accepted observation and synthesises them into a structured product: a cross-validated finding the requester can act on.
The two-model split — Haiku for speed, Sonnet for accuracy — was deliberate. Real-time feedback needs to return in under a second; the final gate can afford to be slower and more thorough.
Challenges we ran into
The validation problem. The hardest part wasn't building the marketplace — it was making the AI validator smart enough to be useful but not so strict it rejected legitimate observations. We went through multiple iterations: keyword matching was too brittle, full Claude calls were too slow for real-time feedback. The solution was a two-tier approach: Claude Haiku for sub-second pre-validation while the user types, Claude Sonnet only on final submit.
Topic drift in mock mode. During testing we found contributors could submit completely off-topic content and pass the validator. We built a domain detection system that identifies the task's category — MRT crowding, cycling infrastructure, rideshare, delivery, traffic — and enforces topic relevance before any other check runs.
PayPal Payouts sandbox behaviour. The sandbox environment has undocumented rate limits and requires specific receiver configurations that differ from production. Getting the payout flow reliable across different test accounts consumed more time than expected.
Making the value legible. The hardest UX problem: how do you tell a contributor their submission was rejected without discouraging them? We added a live feedback panel showing four quality dimensions in real-time — so by the time they hit submit, they already know if it will pass.
What we learned
Building this forced us to think carefully about what AI is actually bad at. The conventional answer is "reasoning" or "maths." The more precise answer is: AI is bad at present-tense, local, first-hand observation. No amount of scaling fixes the fact that a model trained on last year's data cannot tell you which coffee shop in Shibuya has a 40-minute wait this morning.
That gap is permanent and structural — and it's large enough to build a market on.
What's next for UNMADE
The current version proves the core loop — a business posts a question, contributors earn by answering it, AI validates and fuses the results. What comes next is making that loop faster, broader, and trustworthy enough for real money.
Reputation system. Right now any submission is treated equally. The next step is a contributor score — built from acceptance rate, consistency across tasks, and geographic coverage — so high-reputation contributors get priority access to higher-paying tasks.
Verified contributor tiers. For sensitive use cases (financial data, regulatory compliance), requesters need more than AI validation. We want to introduce identity-verified contributor tiers where observations can be attached to a verifiable real-world identity.
Recurring intelligence subscriptions. Most business questions aren't one-time. "Which delivery zones have the worst kitchen wait times?" is a question FreshDash wants answered every week, not once. We plan to support recurring agent deployments — the same question re-run on a schedule, with trend data built up over time.
Expanding the requester side. Right now requesters interact through a simple form. The roadmap includes an API so data teams can deploy agents programmatically, pipe results directly into their dashboards, and set automated triggers (e.g. re-deploy when confidence drops below a threshold).
Beyond text. Observations today are text. The natural extension is structured media — a photo of a queue, a short video of a road condition, a tagged location ping. Multimodal validation opens up an entirely new category of questions that can't be answered in words alone.
The long-term vision is a living dataset layer that sits beneath AI — continuously updated, human-verified, and impossible to fake. Not a replacement for language models, but the ground truth they've always been missing.
