Inspiration
Buying footwear is still largely based on keywords, generic size guides, and shop-floor rules of thumb. But the information that actually matters is how someone moves.
Running and climbing are especially personal: cadence, foot strike, overstride, reach, hip position, and precision can all affect what kind of shoe makes sense. We wanted to make that information usable at the moment of purchase — without requiring a specialist fitting or sending video to a server.
That became Forma: film yourself for 10 seconds, let an agent understand how you move, and get a shoe recommendation backed by evidence you can actually inspect.
What it does
Forma turns a short movement video into an evidence-backed footwear recommendation.
Film 10 seconds of yourself running or climbing and Forma measures movement directly in the browser. It then turns those measurements into fit requirements, researches the live shoe market, checks real retailer stock in your size, and gives you a recommendation with receipts.
Every important claim is designed to be verifiable: research results carry citations, while retailer checks can return a screenshot and session replay showing what was actually found.
Forma also creates a signed Fit Passport containing the user's movement and fitting profile. Other shopping agents can check products against that profile through REST or MCP instead of starting the fitting process from scratch.
After a fitting, the experience can continue through WhatsApp. A Wassist-powered agent can remember the fitting, interpret photos of worn soles, and check in around 12 days later to ask how the shoes are breaking in.
How we built it
Movement analysis runs entirely in the browser using MediaPipe pose tracking, so video does not leave the device unless the user explicitly opts in.
For running, Forma extracts signals including cadence, foot strike, and overstride. For climbing, it looks at reach, hip position, and movement precision.
A Grok gear agent, accessed through Vercel AI Gateway, converts those movement signals into concrete footwear requirements. The agent can then perform live market research through Tavily, with the amount of research controlled by a user-facing Depth dial.
When a user asks Forma to check their size, the system opens the retailer product page in a stealth UK browser through Solari and returns a screenshot and session replay as evidence of the result.
The resulting Fit Passport is signed and exposed through REST or MCP so other shopping agents can use the profile.
The stack includes Next.js, React, TypeScript, MediaPipe, Vercel AI Gateway, Grok, Tavily, Solari, Browser Use, Wassist, WhatsApp, AI SDK, and Three.js.
Challenges we ran into
The biggest challenge was connecting three very different kinds of evidence into one trustworthy experience: movement data, live product research, and real retailer availability.
Movement analysis needed to be useful without pretending that a 10-second video is equivalent to a clinical gait assessment. We therefore focused on concrete, explainable signals that can be measured consistently in the browser.
Live commerce research introduced another problem: a recommendation is only useful if the product can actually be bought. That meant going beyond search results and building a way to verify retailer pages, sizes, and availability while preserving evidence of what the agent found.
We also had to make agentic research understandable rather than magical. The visible research steps and receipts are designed to let the user see where the recommendation came from and inspect the evidence themselves.
Accomplishments that we're proud of
We built a complete loop from movement → fit requirements → live research → verified availability → recommendation → ongoing follow-up.
We're particularly proud that the core movement analysis happens on-device. A user can turn a short video into useful fitting information without uploading the video to a backend.
We're also proud of the evidence layer. Forma doesn't just say "this shoe is available" or "this shoe matches your movement." It can show the source, retailer page, screenshot, and session replay behind the claim.
The Fit Passport extends the idea beyond a single shopping session: a user's fitting information can become portable, signed data that other shopping agents can consume through REST or MCP.
What we learned
We learned that agentic commerce becomes much more compelling when the agent has access to information that traditional shopping interfaces don't.
The interesting part isn't simply asking an LLM which shoe is best. It's giving the agent a new input — how the person actually moves — and then connecting that input to live, verifiable commerce data.
We also learned that trust needs to be designed into agent workflows. Citations, screenshots, replayable browser sessions, and portable fitting data make the agent's reasoning and actions much easier to inspect.
Finally, we learned that privacy can be a product feature rather than just a constraint. Keeping movement analysis in the browser lets the user get useful results while retaining control over their video.
What's next for vertiqal
Next, we want to expand Forma beyond running and climbing into more sports, including cycling and ski boots.
We'll also explore richer wear-pattern analysis from photos, allowing the system to learn from how footwear actually wears over time rather than relying only on an initial fitting.
The bigger goal is to make the Fit Passport a shared layer for agentic commerce. Instead of every shopping agent repeatedly asking users the same questions, a user's fitting profile could travel with them — allowing third-party agents to check products against their real-world movement and preferences.
Ultimately, vertiqal is about giving shopping agents better information about the person they're shopping for, and giving people better evidence about why an agent made a recommendation.
Built With
- ai-sdk
- browser-use
- grok
- mediapipe
- next.js
- react
- solari
- tavily
- three.js
- typescript
- vercel-ai-gateway
- wassist
Log in or sign up for Devpost to join the conversation.