AI-Crew: The Active Retail AI

Concept image of the "AI-Crew" app for in-store usage (featuring a live-action Ami as the clerk).
Development Environment
Agent Management Flow
System Architecture

Inspiration

"We get nervous when store clerks talk to us, but we also want someone to recommend things."

We focused on this common shopper's paradox. In physical stores, human clerks are effective but can create high pressure to buy, often driving customers away. On the other hand, self-checkout kiosks and digital signage lack warmth and serendipity.

We created AI-Crew to solve this by prioritizing "becoming friends" over just "selling." By using a friendly avatar that avoids the uncanny valley and engages in casual small talk (like discussing the weather), we aim to balance labor reduction with high customer satisfaction.

What it does

AI-Crew is an AI avatar sales assistant displayed on store screens that actively initiates conversations and recommends products.

Active Sensing: Instead of passively waiting, it detects customers via camera. When a customer approaches, the AI waves and initiates contact ("Hello! Are you shopping for dinner today?").

Ice Breaking to Sales: It doesn't hard sell immediately. It breaks the ice with small talk ("It's quite cold outside, isn't it?") and naturally transitions to product suggestions ("Since it's cold, we have a great deal on hot pot soup base today!").

Friendly UI: We use a friendly anime-style or 3D character with rich facial expressions to make the AI approachable and fun to talk to.

How we built it

We submitted to the Frankenstein category because AI-Crew is a living creature stitched together from disparate parts to create a unified "AI Kiosk Experience."

The "Body": Standard Kiosk UI We built a Next.js frontend with a clean, focused UI for ordering and payments.

The "Brain": Serverless RAG on AWS Our rag-chat-stream-backend provides a robust API running entirely on AWS.

API Gateway + Lambda (Node.js 20): Implements an OpenAI-compatible endpoint with streaming support.

Vector Search with Qdrant: Manages product data and embeds uploaded Markdown files to cite relevant documents using RAG.

AWS CDK: Defines the entire infrastructure as Code (IaC) for reproducibility.

DynamoDB: Manages agent definitions and conversation logs.

The "Soul": Interactive AI Avatars We integrated AI avatars to provide the face and voice, bridging the gap between the user and the system.

The "Lightning Rod": Kiro Our development was heavily Kiro-driven. We used .kiro/steering to define the project's worldview and .kiro/specs to define technical requirements before writing code. Kiro acted as the "thread" that stitched all these organs together, generating everything from the UI components to the AWS Lambda handlers.

Challenges we ran into

The "3-Tap Paradox": Adding customization options usually increases clicks. We worked with Kiro to brainstorm user flows, strictly deciding what to group together to keep the interface simple.

Conversation vs. UI: Chat is too slow for ordering; buttons are too boring. We found the balance: The AI Avatar handles the "Ice-breaking" and "Suggestions," while the Buttons handle the "Selection" and "Confirmation."

Multimodal Consistency: Keeping the Kiosk UI, RAG Backend, and Avatar in sync was hard. Treating .kiro/steering and .kiro/specs as the Single Source of Truth helped us maintain consistency across these different contexts.

Accomplishments that we're proud of

Successfully Stitched the "Frankenstein": We proved that a combination of AI Avatars + Self-Checkout + AWS RAG + Kiro can work seamlessly together as a unified product.

Kiro-Driven Full Stack Development: We successfully demonstrated that Kiro can handle not just frontend code, but complex AWS backend architecture (CDK, Lambda, Qdrant) and testing (Jest), acting as a "Super-Speed Engineer."

AWS Native Implementation: We built a production-ready, serverless architecture that is scalable and robust, not just a toy prototype.

What we learned

Prompts are Specifications: We realized that a detailed Markdown prompt is more effective than vague instructions. The better the spec in .kiro/specs, the higher quality Kiro's output.

The Human-AI Symphony: The project worked best when we acted as the "Owners of Experience" and Kiro acted as the "Implementer." This clear role division accelerated our development significantly.

What's next for AI-Crew

Personalized Recommendations: Implementing a feature where the Avatar recognizes repeat customers and suggests "The usual" or new items based on purchase history.

Multilingual Support: Auto-switching to English, Chinese, or other languages to support inbound tourists.

POS Integration: Real-time linking to inventory data to prioritize recommending overstocked items or products nearing their expiration date.

Mobile Integration (O2O): Displaying a QR code at the end of a conversation to send coupons for the recommended items directly to the customer's smartphone.

Real-world Deployment: Testing the kiosk in a real pop-up store environment to gather user feedback on the "Avatar interaction" experience.