PhongVu AI Sales Agent

Inspirations

In the retail sector for computers and electronics (such as Phong Vu), customers often struggle with choosing compatible PC hardware, comparing technical specifications, or finding real-time product availability and promotions. Customer support teams are frequently overwhelmed by these repetitive, detail-heavy inquiries.

We were inspired to build a next-generation AI Sales Agent that not only chats via text but can also carry out natural, real-time voice conversations in Vietnamese. The agent is designed to query live inventory, consult on specifications, and seamlessly manage customer shopping carts.

What it does

Phong Vu AI Sales Agent is a comprehensive conversational AI assistant featuring:

Voice-based Sales Consulting: Customers can talk directly to the agent in natural Vietnamese via Agora Conversational AI SDK powered by high-quality Microsoft Neural text-to-speech.
Real-time Database Querying: The agent automatically executes tools connected to Supabase (PostgreSQL) to search products, fetch full specifications, check stock levels, and retrieve active promotions.
Cart & Conversation Management: It automatically adds items to the customer's cart, tracks conversation history, and logs user interaction events for behavioral analysis.
Observability & QA: Integrated with Langfuse to trace every LLM generation and database tool execution, allowing developers to easily monitor prompt quality, debug issues, and calculate token consumption.

How we built it

The system is built on a modern serverless stack:

Frontend & API Routes: Powered by Next.js (App Router) and TypeScript to guarantee strict type safety and high performance.
Database: Built on Supabase (PostgreSQL) to manage products, technical specs, shopping carts, and conversation states.
Voice Channel: Connected to Agora's Conversational AI Agent API to dynamically orchestrate and spawn AI agents into customer RTC call channels.
AI Core: Utilizes Gemini API (via OpenAI SDK) as the primary LLM, and Mimo API (Token Plan) as a highly reliable fallback provider.
Observability: Implemented Langfuse SDK wrapping all model calls and tool executions using unified helper functions traceTool and traceLLMGeneration.

Challenges we ran into

During development, we tackled several complex engineering hurdles:

Asynchronous RTC Channel Integration: Dynamically managing the lifecycle of the AI Agent joining the Agora room, generating RTC tokens with proper privilege expirations.
SDK Type Mismatches: Fixing compatibility errors with the agora-token library, which changed its buildTokenWithUid signature in version 2.x to expect 7 arguments instead of 6.
Unified Tracing Design: Architecting a lightweight middleware wrapper to trace database queries and LLM endpoints without adding overhead latency to live voice communications.

Accomplishments that we're proud of

Successfully deployed a responsive, natural Vietnamese-speaking Voice Agent with intelligent voice activity detection (interrupt mode).
Built a clear and visually intuitive tracing dashboard on Langfuse, capturing input/output payloads and token counts for every execution.
Resolved Row-Level Security (RLS) policies on Supabase tables to keep admin actions secured while enabling fast, public reads for product searches.

What we learned

How to orchestrate and spawn AI agents into live real-time communication (RTC) channels in serverless environments.
The importance of robust token security management and implementing observability/monitoring tools like Langfuse from day one.
How to configure flexible, programmatic fallbacks between different LLM providers to ensure zero downtime.

What's next for Phong Vu AI Sales Agent

PC Builder Compatibility Engine: Build logic to automatically verify hardware compatibility (e.g., matching CPU socket types with motherboards, calculating power supply requirements).
Custom Brand Voices: Train custom text-to-speech models to align with Phong Vu's brand identity.
ERP Integration: Integrate directly with physical stores' warehouse inventory databases for real-time offline stock checks.