-
-
-
Asking for a hoodie
Inspiration
I’m really picky when I shop.
I spend way too much time doing due diligence, comparing options, reading reviews, trying to make the “right” choice. Most of the time I don’t even end up buying anything because there are just too many options.
On top of that, shopping online has gotten worse. There’s so much AI slop now. Fake reviews, agents posting on Reddit recommending random products, and constant targeted ads where you know your best interest isn’t actually being put first.
I wanted something that narrows things down to what I’d actually like without needing to think about it. Just text what I want and get a few solid options grounded in real data, not noise.
Working with Knot made it clear that transaction data is the missing piece. And Photon made it possible to put that into iMessage, where I already am.
What it does
Shomi is a shopping agent you text.
It reads your message, looks at your past purchases, and narrows down options to things you’re actually likely to want. Instead of endless scrolling, it does the filtering for you.
It helps you:
- cut through too many choices
- compare a few relevant options
- understand tradeoffs quickly
- see real products with images and prices
- reorder or check out directly
The goal is simple: take a messy, overwhelming search and turn it into a small set of good options you can actually act on
How I built it
The core is an agent orchestrated with Gemini.
- Photon handles the iMessage interface
- Gemini handles reasoning and tool orchestration
- Knot provides purchase history and payment context and vendor checkout capability
- I used a Walmart scraper for product listings for the MVP
Everything is in TypeScript.
For memory and speed, I implemented three main tools:
Chat memory I summarize conversations and embed them so the agent keeps track of preferences over time. It learns from you and doesn’t forget once context fills up.
Transaction memory I embed Knot transaction history into a vector database so the agent can semantically search past purchases and understand patterns.
Product retrieval I scraped Walmart listings for a few categories, embedded them, and let the agent retrieve relevant products using RAG.
Early on, the agent kept returning the wrong colored products. I fixed that by adding a Gemini multimodal verification step in the tool pipeline to ensure retrieved images actually matched the request before returning them.
Challenges I ran into
The most real one was silly. I spent way too long debugging before realizing Photon’s verification email was sitting in spam.
On the technical side, grounding the agent was harder than expected. Early demos looked fine until small details were off, like product variants or colors.
Fixing that meant adding validation steps and being stricter about what gets returned.
Making it feel real and interactive was a huge part of it as well. Read receipts, varied response times (gemini's true speed feels just plain weird, so artificial delays were introduced to make it feel more conversational). I wanted it to feel like a friend, not just ChatGPT.
Balancing memory vs speed was also tricky. Too much context slows everything down, too little and it forgets what matters. The summarization plus embedding approach ended up working well.
Accomplishments that I'm proud of
The moment it worked felt real.
I texted the agent asking for something we had discussed before. It remembered my favorite color, found relevant items Walmart was selling, then showed me a GIF shifting through all of the options. It also weighed tradeoffs/price values for me as it had understood how important I found value when shopping.
It gave me the same feeling I had writing code when I was younger. That “this actually works” moment. That made me really happy. iMessage APIs are awesome.
What I learned
Good agents aren’t just about calling an LLM.
You need:
- real data (purchase history mattered a lot)
- strong tool design
- validation so outputs are actually correct
I also learned that interface matters more than I expected. Putting this in iMessage made it feel natural in a way a web app wouldn’t.
What's next for Shomi
Right now it’s an MVP focused on narrowing options and helping with decisions.
Next, I want Shomi to act as a full layer between you and your shopping experience.
- automatically create a dedicated agent email
- handle delivery updates and forward them to you
- manage returns using a browser-based agent
- act as a full liaison for anything post-purchase
On monetization, I want to keep things transparent.
The plan is a simple affiliate model where users can opt in to affiliate links. If enabled, some recommendations include affiliate links that support the product.
The important part is that recommendations come first. The system is grounded in your purchase history and preferences, not ads.
As AI systems start introducing ads, trust becomes the most important thing. I want users to know that what they’re seeing is actually in their interest, not just what pays the most.
Built With
- knot
- langchain
- photon
- sqlite
- typescript
Log in or sign up for Devpost to join the conversation.