LiveAssist (Enhanced)

Real Time CS Agent Co-Pilot

Comment 1

Live Assist: Real-Time AI Assistant for Customer Service

Customer service has evolved beyond traditional call handling, now demanding context, speed, and immediate access to critical information. Introducing Live Assist: a desktop-powered AI system that provides real-time call insights, seamless CRM integration, and document-aware responses, all within a robust, serverless AWS architecture. This is not a chatbot; it functions as an agent co-pilot, listening, learning, and acting in conjunction with your team in real time. Live Assist also includes a floating window capability, allowing agents to minimize the application while still receiving live notifications during a call, ensuring continuous awareness and support without interrupting workflow.

Architectural Overview

  • Frontend: Developed with Electron, React, and TypeScript. It captures microphone and system audio, streams it to AWS Transcribe WebSocket, and sends 10-second transcript windows, along with call metadata, to the backend.
  • Real-Time Backend: Utilizes API Gateway and Lambda to receive transcript windows. Lambda retrieves previous windows from DynamoDB to assemble context, then passes this input to a LangChain Orchestrator, powered by Claude via Bedrock. The LangChain agent invokes various tools to provide insights and facilitate actions. For document synchronization, a Lambda endpoint is triggered by S3 document uploads. This Lambda uses LangChain's PDF document loader and text splitter to process documents into text chunks, which are then uploaded to a separate S3 bucket that synchronizes with Kendra. This ensures that the knowledge base is continuously updated and accessible.
  • Post-Call Backend: Aggregates window data by call_id and triggers a post-call Lambda for comprehensive analytics, including sentiment scores, issue resolution, agent performance, and a "Memory Box" for deliverables and coaching points.
  • Core AWS Services: Key services include Transcribe (for real-time transcription), Bedrock (for AI generation via Claude LLM), Kendra (for intelligent document search), DynamoDB (for data storage), S3 (for document storage), API Gateway (for secure API endpoints), SES (for email services), ECR + Docker (for custom Lambda runtimes), and Lambda (for serverless compute).

Agent Intelligence (LangChain-Powered)

The core reasoning capability of Live Assist resides in a LangChain-based agent. This agent receives transcript windows and call history, utilizing a system prompt specifically tailored for customer service support. It retrieves relevant data from CRM and documentation tools, outputting structured tips and tool calls.

Toolkit:

  • get_contact_by_email
  • Contacts_by_phone
  • Contactss_by_company
  • update_contact_by_email
  • search_deals_by_company
  • create_task_for_contact
  • get_company_insights
  • log_call_outcome
  • get_contact_associations
  • search_recent_tickets
  • create_support_ticket
  • send_email (requires confirmation)
  • search_company_manuals

Security & Monitoring

Security measures include the implementation of IAM roles with least privilege, management of all keys via environment variables, and encrypted API requests. Monitoring is facilitated through CloudWatch for logs, metrics, and alerts.

Conclusion

Live Assist represents the future of live agent support, offering real-time insights, context-aware automation, and hybrid action control. This system aims to reduce agent burnout, enhance service quality, and improve customer satisfaction by allowing agents to focus on understanding clients while the AI manages data and provides proactive support.

Share this project:

Updates