Veylo

Inspiration

When people need physical objects in the real world, they rarely know the technical product names:

"I need that little thing that lets me connect my laptop to the TV. I don't know what it's called." "I need the white tape plumbers use around pipe threads." "I need a cable to charge my phone."

Today's voice assistants and search engines break down at this Ambiguity Gap:

Search engines fail because they require precise technical keywords (e.g., "USB-C to HDMI adapter with DisplayPort Alt Mode"). Generic LLM chatbots fail because they hallucinate local store inventory, opening hours, and shelf availability—confidently directing shoppers to closed stores or nonexistent stock.

I envisioned Veylo: an ambient, multimodal shopping intelligence capability built natively for Alexa+. Veylo bridges the ambiguity gap by turning colloquial human descriptions into verified physical-world actions through an inspectable, evidence-first agent loop exposed over the Model Context Protocol (MCP).

What it does

Veylo transforms ambiguous natural speech into concrete local commerce actions without guessing or hallucinating:

Resolves Ambiguous Product Descriptions: Transforms colloquial phrases ("white plumber's tape") into canonical product identities (PTFE thread seal tape) with confidence scoring, functional alternatives, and targeted clarifying questions when needed. Ambient Hardware Context & Redundant Purchase Prevention: Cross-references the user's registered devices (user_devices) and owned items (user_inventory). If you ask for a laptop adapter, Veylo detects your registered MacBook Pro (M3), deduces USB-C compatibility automatically, and warns you if you already own one before you spend money. Evidence-First Trust Model (Zero Hallucinated Stock): Every store recommendation carries an explicit, tamper-proof inventory state (CONFIRMED, LIKELY, WEB_FOUND, UNKNOWN). Veylo's schemas reject any claim of confirmed stock without attached physical evidence. It never fabricates prices, hours, or shelf availability. Interactive Trade-Off Matrix: Renders multimodal comparative cards (e.g., standard PTFE tape vs. gas-line yellow tape vs. thread sealant paste) on the Echo Show canvas, explaining pros, cons, and code compliance. Progressive Live Agent Trace (SSE Streaming): No black boxes. Evaluators and users can watch the agent reason in real time over Server-Sent Events (SSE) as it calls MCP tools, refines hypotheses, and queries local geodata. Authentic Ambient Voice & Luminary Avatar: Streams low-latency spoken responses using Amazon Polly Neural (Joanna) paired with a calm, responsive orbital light physics avatar (idle, listening, thinking, speaking). How I built it

I built Veylo as a production-grade 10-package TypeScript monorepo governed by strict architectural boundaries:

Model Context Protocol (MCP Spec 2025-11-25): The self-hosted MCP server (apps/mcp-server) exposes 11 composable tools (identify_product, discover_local_places, check_local_inventory, compare_products, etc.) over stateless Streamable HTTP. Both the web client and external Alexa+ ambient voice runtimes invoke the exact same MCP contracts. Amazon Bedrock AI Engine (@aws-sdk/client-bedrock-runtime): Powers zero-shot semantic hypothesis generation and multi-product trade-off evaluations via the modern unified ConverseCommand API (Claude 3.5 Sonnet and Amazon Nova Pro), constrained to maxTokens: 1024 for sub-second ambient latency. Amazon Polly (@aws-sdk/client-polly): Synthesizes low-latency audio via the Joanna Neural engine, matching Alexa's authentic voice persona, with automatic graceful fallback to system natural speech. Amazon RDS / Aurora PostgreSQL Serverless v2: Powers relational taxonomy graphs, pg_trgm trigram similarity matching for phonetic and typo tolerance, and cross-session user hardware context memory. Next.js 15 & Tailwind CSS: Delivers a calm, restrained Notion/Linear-inspired interface simulating an Echo Show 10 multimodal canvas with zero visual clutter. OpenStreetMap Spatial Discovery: Queries real-world retail coordinates using live Overpass spatial queries and Nominatim geocoding. Challenges I ran into Preventing Agent Loops & Runaway Latency: Ambient voice interactions demand answers in under 2 seconds. I engineered a bounded agent loop capped at a strict 8-step budget with dynamic early termination as soon as high-confidence evidence is verified. Bedrock Cross-Region Profile Resolution: Configuring on-demand foundation models in AWS Bedrock required discovering and standardizing on cross-region inference profiles (us.anthropic.claude-sonnet-4-6) to prevent regional capacity limits. Zero-Crash Evaluation Resilience: Knowing that judges may clone the repository without AWS credentials, I engineered transparent fallback systems: Bedrock degrades cleanly to offline deterministic rules (meta.fallbackFrom = "bedrock"), and Polly degrades to browser natural synthesis, maintaining a 100% working demo guarantee. Streaming Web Streams vs. Node Streams: Next.js App Router route handlers require standard Web BodyInit streams. I bridged AWS SDK v3's SdkStream into typed ArrayBuffer byte slices for smooth audio chunking. Accomplishments that I'm proud of 100% Protocol Compliance: Passed 8 out of 8 automated conformance checks against the latest MCP Specification (2025-11-25) using official @modelcontextprotocol/sdk tooling. Zero Hallucinated Stock Policy: Built an architectural barrier where generative AI models are strictly forbidden from fabricating physical store availability—relegating inventory truth entirely to real-world geodata and verified evidence ledgers. 134 Automated Tests: Maintained 100% test pass rate across unit, integration, and typecheck suites with zero build warnings. Sub-Second Ambient Voice Response: Integrated Amazon Polly Neural and Bedrock token management to achieve snappy, human-sounding ambient conversations suitable for Echo Show smart displays. What I learned Bounded Agents Outperform Unbounded Prompts: Single monolithic LLM prompts frequently hallucinate local commerce details. Bounding the model into discrete, observable tool calls with hard step budgets yields vastly higher accuracy and user trust. The Power of the Model Context Protocol: Decoupling tool definitions into MCP contracts allowed my Next.js frontend, test runners, and simulated Alexa+ voice skills to share one unified intelligence backend with zero duplicate code. Modern AWS SDK v3 Modularity: Utilizing modular packages (@aws-sdk/client-bedrock-runtime, @aws-sdk/client-polly) kept my production bundles lean and enabled cross-model portability with zero prompt re-engineering. What's next for Veylo Multimodal Echo Show Camera Vision: Enabling Alexa+ devices with cameras to identify unlabeled cords, stripped screws, and broken plumbing fittings simply by holding them up to the display ("Alexa, what is this and where can I buy one nearby right now?"). Direct Merchant POS Integrations: Partnering with regional retailers and home improvement centers to ingest real-time POS stock webhooks and aisle/shelf-level item location. One-Shot Curbside Reservation: Enabling Alexa+ to not only find the verified item at a nearby store, but immediately reserve it for curbside pickup via Amazon Pay with a single voice confirmation.