Inspiration
Small industrial distributors and wholesalers still run order intake almost entirely by hand — a WhatsApp message in Hindi, a phone call, a photo of a handwritten order sheet. Every one of those has to be manually read, translated, matched against a product catalog, checked against a customer's credit and delivery terms, and finally typed into billing software before anything ships. That gap between "the customer told us what they want" and "the order is a structured, billable record" is where mistakes, delays, and mistrust creep in. We wanted to see whether a single agent could sit in that gap and turn any channel — text, voice, or image — into one clean, auditable order, while still keeping a human in the loop for anything risky.
What it does
Valence is an order intake and fulfillment agent built on Google ADK. It listens on WhatsApp, a recorded phone line, and photographed order sheets, and in any language, extracts a structured order: customer, items, quantities, and delivery location. That order is handed to an Order Processing Core — a decision engine that either auto-approves clean, low-risk orders or escalates anything uncertain (an unverified number, an uncatalogued product, a missing field, low model confidence, an unusually large order, or an anomaly) to a human reviewer through a passcode-gated web queue. Approved orders get a two-tier draft price estimate and, downstream, generate the Loading List and Tally billing artifacts the business already uses. Every decision — auto-approved or escalated — is written to an audit trail so nothing happens invisibly.
How we built it
The core is a Python service on Cloud Run, built around a single Google ADK agent backed by Gemini through Vertex AI. Each sender gets a durable session stored in Firestore, so a Cloud Run restart never drops an in-flight conversation. The agent has exactly one tool exposed to it — process_order — which routes into the Order Processing Core; the phone number that tool trusts is pinned to the caller's authenticated session identity rather than anything the model extracts from message text, so a compromised or clever prompt can't mint an order under someone else's name.
Each channel is a thin, swappable adapter behind a shared seam: a Meta Cloud API adapter verifies WhatsApp's webhook signature and parses inbound messages before anything touches the agent; a voice-ingest endpoint takes company-recorded call audio with the caller identity supplied as trusted, token-authenticated metadata rather than caller ID; photographed order sheets go through the same media pipeline as any other input. All three converge on one run_turn path, so the decision logic, audit trail, and review queue are identical no matter how the order arrived.
The review web view is served from the same FastAPI app, gated by a per-deployment passcode, and shares the exact same approval function as the WhatsApp approval flow — so a human never has to worry about the two paths drifting out of sync. The whole GCP stack (Cloud Run, Firestore, Cloud Storage for vouchers, Pub/Sub for late-order cutoffs, IAM, and Secret Manager for every credential) is stood up by a single provisioning script, and deploys auto-trigger on push to main via Cloud Build.
Challenges we ran into
Trusting identity correctly across three very different channels was the hardest design problem: a phone number typed into a WhatsApp message, a caller ID on a recorded line, and a name scrawled on a photographed order sheet all carry very different levels of trust, and conflating any of them with the model's own extraction would let a bad actor talk their way into someone else's account. We resolved this by treating identity as something that always comes from an authenticated channel boundary — never from the free text the model reads — and wrote that down as an explicit architectural decision so it couldn't quietly regress later.
Keeping the agent's behavior trustworthy enough to demo live was the other big one. Language model output is inherently a little unpredictable, and an order-taking agent that occasionally invents a product or silently drops a field is worse than useless. We built a 24-case eval harness that drives the real agent end-to-end and grades it on the decisions and tool calls it actually made — not on exact wording — so we could catch regressions in accumulation logic, money-policy edge cases, and prompt-injection attempts before they showed up in front of a customer.
Accomplishments that we're proud of
Getting one agent to genuinely unify three very different intake channels — text, voice, and image — into a single structured, auditable pipeline, without those channels leaking their quirks into the core decision logic, feels like the right shape for this problem. We're also proud of the eval harness: pinning the clock so time-sensitive cases (dedup windows, late-order cutoffs) are reproducible, and asserting that risky tools are never invoked on a safety case, gave us real confidence rather than a demo that just happened to work once.
What we learned
Building an agent for a real operational workflow forced us to think less about "what can the model do" and more about "what should never be left to the model" — identity, money thresholds, and anything that could touch another customer's data belongs firmly outside the model's discretion, expressed as code and config rather than prompted behavior. We also learned how much a shared decision core buys you: once every channel funnels into the same Order Processing Core and the same audit trail, adding a new intake surface becomes a boundary-adapter problem instead of a redesign.
What's next for Valence
Next up is finishing the review web view's editing tools — letting a human correct order fields, override GST, and resolve unmatched products inline rather than only approving or rejecting — plus wiring in the Cutoff chain for automatically flagging and dispatching late orders. Longer term, we'd like to extend the same core to handle partial fulfillment and multi-vendor sourcing, so the agent isn't just taking orders faster but actively helping fulfill them.
Log in or sign up for Devpost to join the conversation.