Inspiration
PayPal already owns Venmo — the default answer to "who owes who" for an entire generation. But Venmo can only do one thing: move money a human has already decided to move. It has no judgment. The hackathon's own framing — "how people build, buy, sell, work, connect, and move money" — pushed us toward a bigger question: what if the agent didn't just relay a payment a human typed in, but actually decided what to collect, when to nudge, what to buy, and when to stop and ask? That's the gap between "payment automation" and real agentic commerce, and it's the gap Kitty is built to close.
What it does
Kitty is an autonomous shared-treasury agent for groups — roommates, trip groups, office teams. It runs two real money-moving flows, both driven by an AI agent, both settled through actual PayPal sandbox transactions:
- Collect: upload a photo of a receipt. The agent itemizes it with Gemini vision, asks a clarifying question only when an item is genuinely ambiguous (a shared appetizer, a bottle of wine), computes each person's proportional share of the subtotal, tax and tip, and autonomously creates and sends real PayPal invoices via the Invoicing API. It polls for payment, sends at most two automated reminders, then stops and flags a human instead of nagging forever.
- Spend: each group has a pooled "kitty" with a hard spend cap. When triggered — a photo of a depleted supply, a chat request, a schedule — the agent searches Channel3's live product catalog, and if the item is within budget, autonomously completes a real PayPal Orders purchase from the pooled funds. Cross the spend cap, and it halts and requires explicit human approval before spending a cent more. Every decision the agent makes — every tool call, every dollar — streams into a live reasoning panel and an AG Grid operational ledger, so the autonomy is auditable, not a black box.
How we built it
- Frontend: React 19, TypeScript, Tailwind CSS, AG Grid Community for the ledger, Lucide icons.
- Backend: a single Express app handling both the API routes and, in dev, Vite's SPA middleware.
- Payments: PayPal Sandbox — OAuth2 client-credentials flow, the Invoicing API (create/send/remind) and the Orders API (create/capture) called directly, with every call logged into the reasoning stream as a real tool call.
- AI: Gemini vision (
gemini-3.5-flash) does the actual receipt/shelf-photo itemization; Groq is wired in as a parallel vision path for redundancy. - Commerce: Channel3's product search API powers the autonomous Spend flow.
- Deployment: the same codebase runs two ways — a persistent Node process (for local dev and Render) and a Vercel serverless entry point (
api/index.tswrapping the Express app, with the Vite build served statically) — live at kitty-treasury-agent.vercel.app.
Challenges we ran into
- PayPal's Invoicing scope doesn't grant instantly. Enabling "Invoicing" on a sandbox app's Features page shows as checked immediately, but the actual OAuth scope lags behind — we only found this by directly inspecting the token's
scopefield before and after toggling the feature, across two separate sandbox business accounts, before confirming it was a propagation delay rather than a permissions or region bug. - Invoice payload validation. PayPal rejected a naive
tax_totalbreakdown that didn't reconcile with per-item tax figures. Fixed by folding tax and tip into their own explicit line items instead of a manual amount breakdown. - Groq had no vision-capable model available on our account — we only discovered this by enumerating every model the API key could actually see. Gemini became the real vision path instead, including a live mid-build swap when the originally-coded model name (
gemini-2.5-flash) was deprecated for new accounts. - Making one codebase work both as a persistent server and as Vercel serverless functions without breaking local dev — required splitting the Express app export from its
listen()call and adding a dedicated serverless entry point. - Cross-platform
npm start: inlineNODE_ENV=productionsyntax silently breaks under Windows' defaultcmd.exescript shell; fixed withcross-envso judges on any OS can run it from the README.
Accomplishments that we're proud of
- Every integration is genuinely live, not mocked: real PayPal invoices, real PayPal orders, real Channel3 product search, real Gemini vision — each one verified end-to-end against the actual sandbox and the live production deployment, not just a local happy path.
- Guardrails that mirror the trust posture PayPal itself cares about: a hard spend cap that requires human approval, and a reminder policy that caps itself at two autonomous nudges rather than escalating indefinitely.
- A visible, honest reasoning stream — when something fell back to a simulated response during development, it was clearly labeled as such rather than faked, and we kept pushing until every core flow was real.
What we learned
- Sandbox environments lie about readiness in subtle ways — a feature toggle that looks enabled isn't the same as a token that actually carries the permission. Always verify by inspecting the real token, not the dashboard checkbox.
- The guardrails turned out to be the most interesting design work, not the plumbing — deciding when an autonomous agent should stop and ask a human is a harder and more important problem than making it technically capable of acting.
- Honesty compounds: labeling simulated behavior clearly during development, instead of quietly shipping it, is what kept us finding and fixing the real integration bugs instead of hiding behind a convincing fake.
What's next for Kitty
- Replace in-memory ledger state with a persistent store (Postgres/Supabase) so treasury state survives serverless cold starts reliably.
- Expand autonomous Spend triggers: recurring subscription renewals, inventory-threshold restocking from a shared pantry photo.
- A fully chat-native flow over WhatsApp/SMS — receipt photo in, payment requests out, no app required.
- Move from direct PayPal REST calls to the official PayPal Agent Toolkit / MCP server for deeper, more standardized agentic tool-calling.
- Multi-currency and multi-region support, matching PayPal's own global reach.
Built With
- ag-grid
- agentic-commerce
- ai-agents
- channel3
- express.js
- finance
- gemini-vision
- google-gemini
- groq
- llm
- node.js
- paypal
- paypal-invoicing-api
- paypal-orders-api
- paypal-sandbox
- react
- tailwindcss
- typescript
- vercel
- vite