Landing Page
Sign-in Page
Buyer's Dashboard
Contributor Dashboard
Skarl Mobile App
Mobile Video Uploading Workflow
Video accepted and paid out
Agent functionality
Buyer Finance Agent
App Integrations
Footer

Inspiration

Robots are learning to use their hands from videos of people doing everyday tasks, and companies are paying real money for that footage. In August, Figure said it plans to spend a billion dollars on data and compute over the next year, and it has already paid contributors $15 million through its own gig app.

When we read the App Store reviews for apps in this space, most complaints were about money: payouts sitting on pending for weeks, and clips rejected with no reason given. Buyers have their own problem. One data vendor estimates that around 90% of bulk footage gets thrown out for any specific robot task, and nothing stops someone from uploading the same clip twice under a new filename. We wanted to build the piece in the middle, which checks whether the work actually happened, pays for it as soon as it does, and leaves a receipt that the buyer and the person who filmed it can both read.

What it does

Skarl has two user flows: one is a web console for a company buying video data, and another is a phone app for the person recording the egocentric footage.

On the web, a buyer posts a task with plain-language checks, like "pick up the can" and "place it closer to the hat next to the lamp," and sets how many clips they want and what each one pays. They fund the order in USDC on Base through x402, either from Coinbase Wallet or by letting their own AI agent pay the endpoint directly. While funding, they set an auto-pay rule. The default releases payment only when every check passes and keeps each payout under the per-clip price. Every submission shows up in their console with timestamped evidence for each check, and anything that fails or looks off lands in a review queue along with the reason. A finance agent watches the books and drafts payments, but it can't approve its own, and a Coinbase wallet cap sits behind every transfer as a second limit. The ledger page lines up the order, our double-entry books and the chain, so the buyer can see the money matches. Accepted clips export as a LeRobot v2.1 dataset, with the contributor's hand keypoints mapped to joint targets for an SO-101 robot arm, so a robotics team can load them straight into training.

On the phone, a contributor signs in with Google, picks a task and sees exactly what it pays before filming. Once they agree to the license terms, the camera opens with each step on screen, and they narrate as they go and read a few random words aloud so we know it's a live take. The phone also records motion data alongside the video. After upload, Gemini watches the clip and separately listens to the narration, and a copy check compares it against earlier submissions so the same clip can't get paid twice. The contributor can follow each take from review through payout. Their payout wallet lives in their profile and gets created on the phone with a single tap, with no seed phrase to write down.

How we built it

Skarl is a pnpm monorepo split into three parts. The buyer console is built with Next.js 16 and Tailwind, the contributor app uses Expo and React Native, and Convex is the backend underneath both. We chose Convex because the money side needed transactions we could trust, and every amount stays an integer count of micro-USDC from the database to the API, since a rounding error in a payout is a real bug.

Verification starts on the server. FFmpeg pulls frames and audio out of each clip and hashes them; the frames go to two vision models that have to agree, and when they don't, the clip goes to a person. The audio gets transcribed separately, so the narration gets checked against what the camera saw instead of being trusted on its own. Copy detection compares frame hashes with an alignment step, which means a trimmed or re-encoded copy of an earlier clip still gets caught. Each finished report is attested on Base Sepolia through the Ethereum Attestation Service, leaving a public record of the verdict without publishing anyone's video.

On the phone, the camera records the clip while the motion sensors capture a trace alongside it, and the app shows three random words for the contributor to say on camera so we know the take is live. Sign-in uses Convex Auth with Google. The payout wallet is a key generated on the device and kept in the iOS keychain, and it proves ownership by signing a one-time message that the backend verifies before any money can go to that address. Agents can also find Skarl on their own through an MCP server and an llms.txt file.

Hyphenate. Skarl does two finance jobs, settlement accountant and accounts-payable clerk, and the rule we built everything around is that the AI that drafts a payment can never approve it. The finance agent proposes, the buyer's signed auto-pay rule or the buyer themselves approves, and a separate wallet cap enforces a hard limit even if both of those go wrong. A copied clip can't create a second payable, and when a check fails, the case goes to the buyer with the reason attached so nobody has to guess.

Coinbase. All the money on Skarl moves on Coinbase's stack. Buyers fund orders in USDC on Base over x402 through the CDP facilitator, and signed CDP webhooks confirm each transfer before it reaches the books. Payouts come from a CDP wallet whose policy caps every transfer and only allows contributor addresses verified by signature; during testing it really did reject an over-cap send and a send to an unknown address. We also wrote a plugin for Coinbase Wallet MCP, so someone's own AI agent can create and fund a Skarl order while the person approves in their wallet. A personal agent can also onboard itself in one call through our MCP tools and skill file, whether it's buying data or doing the work, and a rate-limited faucet gives it testnet USDC to start.

Convex is our system of record. Orders, uploaded clips, verification reports, approvals and every journal entry live in it, and because its mutations are transactional, accepting a clip creates the delivery, the payable and the balanced ledger entries together or not at all. The scheduler and crons run verification and attestation jobs, the Workflow component holds a payment while it waits for approval, and live queries keep the buyer console and the phone app in sync without polling.

Gemini Argon and Flash are our video auditor. They look at frames from each clip without hearing the audio and have to agree; if they don't, a person reviews it. Gemini's transcription model handles the narration on its own, so we can catch someone describing a task the camera doesn't detect, and Gemini also runs one of our two finance agents.

Claude runs the other finance agent, with 24 strictly typed tools over the ledger: it answers why a clip was paid or held with cited tool calls, drafts payouts and writes exception notes, but it can't approve or sign anything, and it treats every tool result as untrusted data. It also powers a coach that tells contributors why a clip got rejected, using the stored evidence, and it has no way to change the verdict.

Capital One's Nessie API gives us a mock bank to reconcile against. For a 10 USDC order, the order terms, our ledger, the chain and the bank record all matched. It's a sandbox bank, so no real dollars moved.

Atlassian. We built Skarl Guard, a Forge app for teams that let agents work in Jira. It triages issues, keeps anything that reads like an instruction to the agent in comments for a person to review, caps how many actions an agent can take, logs each one, and can undo a change only if nobody has edited it since.

Solana is where a buyer's purchase becomes something they can prove. Every delivery Skarl publishes mints a one-of-one Token-2022 license on Solana. Supply is fixed at 1, the mint authority is revoked in the same transaction, and the token's on-chain metadata carries a hash of the dataset manifest, the license terms and each contributor's consent receipt. A buyer, an auditor or a robotics lab three steps downstream can open the token and check exactly what was licensed and who agreed to it, without needing to trust us.

We picked Solana because Token-2022 let us do this without writing a custom program. Its built-in metadata extensions give every license its own on-chain record, and a mint costs about 0.006 SOL, so minting one per delivery is cheap enough to do every time. The mint is signed remotely through CDP with a fresh key, so no wallet key is ever exported.

Solana also works as a funding rail. Buyers can pay for an order in USDC on Solana devnet through the same x402 flow we use everywhere else. We ran a 1 USDC funding end to end: it finalized on chain, and our Convex ledger matched it to the exact micro-USDC with zero reconciliation exceptions.

Challenges we ran into

Keeping the buyer console, the contributor's phone and the chain all in sync was much more difficult than we expected. So Convex is the only place anything gets decided: the phone and the web read the same records over live queries, and a transfer only counts once a signed CDP webhook and an on-chain lookup agree with the payment we already recorded. Getting there meant tracking down drift we created ourselves. At one point the phone was pulling a task's recording steps from an old take saved on the device rather than the task the contributor had just opened, and the task list was resubscribing to the backend on every render until React stopped it.

Wallets attached to a device were harder than they sound. The key gets generated on user's device and stays in the iOS keychain, so the backend never sees it, which means ownership has to be proven instead of assumed. The server issues a one-time sign-in message tied to our domain, the Base Sepolia chain and that contributor's membership, the phone signs it, and the server checks the signature before the address can ever receive money. Our first live check failed even though the same signature verified fine on a laptop: the signing library loads its curve code lazily, and Convex mutations can't do that kind of import, so we bundled the curve ourselves. We also had to make sure a phone can't quietly replace a wallet that's already verified, since the payout allowlist is tied to that exact address.

Also, our video verifier started out knowing only one task. We built and tuned everything around a red cup going onto a coaster, so the vision prompts and the report format only had words for cups and coasters. When we added a second task in the early morning (moving a can closer to the hat next to a lamp), the checker had no way to describe a can, so we reworked it to build its checks from each task's own rubric.

Accomplishments that we're proud of

Real money moved through the system, on testnet but over the actual rails. A buyer funded a 10 USDC order over x402, a second funding went through Coinbase's facilitator with a signed webhook, and in both cases our ledger balanced to the micro-USDC and matched the chain. For that 10 USDC order, the bank record lined up too.

We're proud that the safety limits held up when we tried to break them. When we asked the payout wallet to send more than its cap, and then to send to an address it didn't know, Coinbase refused both. The finance agents ran live against the real ledger on Claude and on Gemini, explained what they saw, and proposed nothing they weren't allowed to.

Every finished verification report is attested on Base Sepolia, eleven so far, so anyone can check that a verdict existed and hasn't changed without ever seeing the contributor's video.

And it's a whole product with a buyer console where you post, fund and review tasks, a phone app where someone can sign in with Google, pick a task, record it and create a payout wallet with one tap, and a backend that maintains alignment.

What we learned

Most of what makes a payment system trustworthy has nothing to do with sending money. Sending a transfer took an afternoon. Knowing whether it actually happened, and making sure a retry, a timeout or a duplicate upload could never pay someone twice, took most of the weekend, and we ended up treating every payment as a claim that stays open until the chain confirms it.

We learned to be careful about what counts as a second opinion. Two Gemini models agreeing on a clip is more useful than one, but they come from the same family and run on the same infrastructure, so they can be wrong together; that's why a person still reviews anything they disagree on, and why we kept the copy check and the ledger rules completely separate from the AI.

Separating who proposes a payment from who approves it changed how we designed everything else. Once the AI agent was only allowed to draft, we could let it be useful (reading the books, explaining a mismatch, proposing a payout) without worrying about what it would do with more access.

What's next for skarl.

We want to know how often our video checks are right. That means collecting a set of consented, hand-labeled clips and measuring how many bad takes we wrongly accept and how many good ones we wrongly reject, instead of only showing that the pipeline runs.

On the phone, the payout wallet should survive a lost phone, so we'd move from a key that lives only on the device to Coinbase's embedded wallets, which tie the wallet to the contributor's sign-in. We also want richer recordings that robotics teams actually ask for, like hand pose, depth and camera position from ARKit, plus a head-mounted mode for first-person footage.

Longer term, the plan is mainnet USDC, a way for contributors to cash out to a bank account, and a pilot with a small human-data company to find out whether evidence-backed payouts make their reviews and disputes easier.

We also want richer recordings that robotics teams ask for, like depth and camera position from ARKit and a head-mounted mode for first-person footage, and to take the SO-101 export from advisory joint targets to actions validated on a real arm.