EnergyCopilot

Inspiration

The idea for EnergyCopilot came from a real-world pain point: energy contracts and technical documents are long, dense, and difficult to interpret—especially for non-experts. I wanted to build a tool that could help people ask questions in natural language and get fast, reliable answers grounded directly in those documents. With the recent advances in large language models (LLMs) and serverless architecture, I saw an opportunity to make this real-time, scalable, and cost-efficient.

What it does

EnergyCopilot answers user questions about energy documents in real time using retrieval-augmented generation, semantic search, and token-level streaming through a WebSocket interface.

How we built it

A React + Tailwind frontend connects to the backend via API Gateway WebSocket
An entry-point Lambda function handles connection tracking and pushes messages into SQS
A containerized Lambda function handles RAG inference using LangChain and Qdrant, returning grounded responses in real time
A semantic cache checks for similar past questions to avoid redundant computation
Responses are returned token-by-token over WebSocket for a smooth, chat-like experience

Challenges we ran into

WebSocket communication debugging
Ensuring stable bidirectional messaging and correct connection tracking across Lambda, API Gateway, and DynamoDB required careful handling and retry logic.
Cold start optimization
Managing startup latency for container-based Lambda functions, especially those loading local models and vector indexes, was essential for maintaining responsiveness.
Frontend rendering and streaming
Fine-tuning token-level rendering and scroll behavior in the React frontend to provide a smooth streaming experience.
Balancing recall and semantic precision
Designing the semantic cache to tolerate minor variations in phrasing while avoiding irrelevant matches was a constant tradeoff between coverage and accuracy.

Accomplishments that we're proud of

Built a fully serverless, real-time Q&A system combining LLMs with retrieval-augmented generation
Implemented token-by-token streaming over WebSocket for a smooth, chat-like experience
Designed a semantic caching mechanism that reduces latency and OpenAI API usage by up to 80%
Deployed a scalable and decoupled architecture using AWS Lambda, API Gateway, SQS, and DynamoDB
Achieved reliable multi-user communication with persistent WebSocket connection tracking
Integrated Qdrant for fast, accurate vector search to support semantically grounded responses

What we learned

Retrieval-Augmented Generation (RAG)—how to integrate document retrieval with LLM-based answer generation
AWS Lambda, API Gateway (WebSocket), SQS, and DynamoDB for building event-driven serverless systems
Qdrant-based semantic caching—reducing latency and API usage by reusing answers to similar questions
Streaming architectures using WebSocket for responsive user interaction