Inspiration
Traditional customer support is slow and repetitive
Customers often wait too long or repeat information multiple times
We wanted to build an intelligent system that automates ticket analysis, identifies similar issues, and escalates only when necessary
The idea was to combine LLMs, vector search, and voice AI into one streamlined escalation system
What it does
Accepts support tickets through an API
Uses Amazon Bedrock to analyze and classify the issue
Creates vector embeddings and checks for similar past tickets
If a similar solution exists, it generates a response with a confidence score
If the confidence is low or no match is found, it triggers a voice call to a Level 1 agent using Twilio
Conducts a natural, multi-turn conversation to gather more information
Logs and stores all responses in MongoDB
How we built it
Flask-based backend handling REST API and Twilio webhooks
NLX for the MCP server and Claude desktop for the MCP client
Amazon Bedrock for issue classification and AI-generated solutions
SentenceTransformers to generate ticket embeddings
S3 for storing embeddings and MongoDB for storing all ticket metadata
Twilio Voice API with Polly voices for natural speech and Gather for speech input
Thread-safe memory handling for active calls and their state
Challenges we ran into
Making Twilio voice flows sound natural while handling no-input or unclear responses
Thread-safe handling of concurrent voice calls and live response data
Embedding generation and retrieval with consistent similarity scoring
Integrating Bedrock, embeddings, and Twilio smoothly into a real-time pipeline
Managing multiple webhook callbacks during and after the voice interaction
Accomplishments that we're proud of
Built an end-to-end smart escalation system from ticket submission to voice support
Integrated vector search and LLM-based solution generation
Created a voice flow that dynamically adjusts based on responses
Automated the decision to escalate based on confidence scores and past tickets
Successfully launched real phone calls and tracked responses in real time
What we learned
How to build a scalable AI-driven support system
Effective use of Bedrock and vector similarity in real-world flows
How to handle speech input, call states, and fallbacks in Twilio
The value of voice-based interfaces in support escalation
How to structure threaded voice call processing with shared state
What's next for AgenSupport
Add a dashboard for agents to view active calls and ticket histories
Use LLMs to summarize multi-turn call transcripts
Add support for other channels like SMS or Slack
Introduce learning loops from past resolved tickets to improve matching
Let customers directly interact with the AI before any human escalation
Log in or sign up for Devpost to join the conversation.