Aria

We help blind people shop fashion with confidence!

Comment

Inspiration

Clothes are personal. They are one way we decide how to show up in the world.

For a blind shopper who relies on family or friends to browse clothes, a shopping trip can mean asking someone else to describe the options, check what matches, and help make the choice. That support can be valuable. But it should be a choice too.

We are building Aria so a person can explore their own style, understand their options and decide with more confidence. Why should someone have less say in what they wear because the information is presented visually?

Our first version read the screen aloud. During development, we realised that this was not enough. A companion needs to understand the next step, explain what matters and wait for the person's decision. That changed the entire journey.

What it does

Aria starts by asking what support would help: spoken guidance, speech on request, quiet mode for a screen reader, larger text or higher contrast. It asks about preferences, not a diagnosis.

The wardrobe stores familiar names, tactile identifiers and physical locations. Aria can suggest an outfit from confirmed garments and help locate them. Unknown colours stay unknown until confirmed. Saving a recommendation requires a separate decision.

Shopping begins with what is already owned. Aria explains an existing pairing before offering an optional gap to explore. A fresh Shopify cross-store search finds actual products, presents a few offers one at a time, and explains price and available options. The user chooses the exact variant, reviews it, and confirms a merchant handoff. Aria does not place an order.

How we built it

The app runs on AWS. Cognito handles sign-in with PKCE; API Gateway validates tokens; Lambda serves the application and derives the storage identity from the verified user. DynamoDB holds wardrobe state and asynchronous jobs. Conditional writes protect against stale or duplicate updates.

A background Lambda runs a Strands agent. Its read-only wardrobe_evidence and find_my_garments tools ground the recommendation in deterministic colour pairings and actual garment locations. OpenAI GPT-4.1 mini is the active Strands model provider. The optional photo specialist uses the OpenAI Agents SDK. Amazon Polly speaks the current companion prompt instead of reading page content. CloudFormation, S3, Parameter Store and CloudWatch support deployment and operation.

Shopify Global Catalog MCP supplies live discovery and exact-variant lookup. We refresh the chosen variant and price before providing a merchant link. Search receives a generic item query, budget and country, not personal garment names or locations.

The architecture includes an optional Bedrock/AgentCore path, but AgentCore is not active in the recorded deployment. Sarvam and Myntra are not connected.

Challenges we ran into

A voice that reads everything creates more work for the listener. We redesigned speech around one prompt and one decision, with persistent pause and quiet mode.

Commerce also exposed a less visible problem: looking up a grouped product could return its default colour instead of the colour shown in search. We anchored the journey to the exact variant and block substitutions. We also fixed the public discovery profile's cache headers and payment-handler declaration so real Shopify requests could run.

Colour compatibility is only one piece of a clothing decision. It cannot establish fit, tactile comfort, personal style or delivery eligibility. Aria makes those limits visible instead of presenting a colour score as certainty.

Accomplishments that we're proud of

The demonstration uses three clearly labelled synthetic garments and real execution: a fresh Strands job, wardrobe tool calls, an explicitly saved recommendation, live product discovery and a real merchant checkout handoff. The recorded checkout contained one dark-red trouser, size 6, regular length, at ₹1,600. We stopped before entering contact details or paying.

The published source includes 40 passing backend tests, speech lifecycle checks, deterministic engine parity fixtures, infrastructure, setup instructions and an editable architecture diagram. The demo is 3 minutes 45 seconds, assembled from actual browser captures with narration and condensed waits.

What we learned

Independence is not the same as doing everything automatically. A useful agent can make information easier to understand while keeping decisions with the person. “You already have something that works” can be a better answer than another purchase.

What's next for Aria

Co-design with blind shoppers is the next essential step. This is a working pilot, not a completed blind-participant study. We want to measure independent task completion, recovery from mistakes and confidence, then improve the journey from what people actually need. Consented voice input, stronger size and returns support, and merchant accessibility work follow from that learning.

The cloud application incorporates our earlier Kasane/GapFinder colour engine and an attributed 86-entry Wada working subset. Those components are disclosed in the repository; we do not claim to have invented the colour theory. AI coding assistance was used.

The choice should stay theirs.

Built With

Share this project:

Updates

Submission history