rfc-rag-agent

• 

  RFC-RAG-Agent answers engineering questions with tool calling, hybrid retrieval, traceable evidence, and citations.

• 

  RFC-RAG-Agent indexes 1,153 engineering documents into 51,738 searchable chunks with direct access to original sources.

Inspiration

Technical knowledge is often scattered across long documents, tables, figures, development notes, and evaluation records. I wanted to build an agent that could turn this fragmented information into reliable, grounded answers instead of simply generating plausible text.

What it does

rfc-rag-agent is a modular RAG agent for complex technical knowledge bases. It routes questions to specialized tools, retrieves supporting evidence from text, tables, and figures, supports image-based analysis, provides citations, and explains clearly when a request is outside the project’s scope.

How we built it

We built the project as a FastAPI-based agent system with a modular tool-calling runtime. The runtime separates coordination, typed tool contracts, tool registration, execution, result merging, refusal handling, and final answer generation. We also added repeatable evaluation cases to validate retrieval quality, answer behavior, and end-to-end latency.

Challenges we ran into

The biggest challenge was balancing modularity, answer quality, and latency. An early modular version made the tool-calling chain heavier and introduced visible latency, especially for figure-related questions. We had to slim down the runtime, reduce unnecessary tool calls, and preserve clear boundaries without adding avoidable orchestration overhead.

Another challenge was refusal behavior: a production-oriented agent should not merely return an “off-topic” label. It should explain what the project can help with and guide the user toward relevant questions, while avoiding exposure of raw provider responses, private logs, or retrieved chunks.

Accomplishments that we're proud of

We transformed the tool-calling path into a more maintainable modular architecture while keeping the actual runtime chain lean. The agent can handle multiple knowledge modalities, provide grounded citations, and give user-friendly scope guidance. We also established a fixed, reusable evaluation set for regression checks across text, table, figure, and image-oriented questions.

What we learned

We learned that modularity is valuable only when it improves maintainability without unnecessarily increasing user-visible latency. We also learned that agent quality is more than retrieval accuracy: tool selection, safe output handling, refusal UX, observability, and repeatable evaluation are all essential parts of a reliable system.

What's next for rfc-rag-agent

Next, we will expand the fixed evaluation set, continue optimizing latency for multimodal and figure-related queries, improve tool-selection policies, and strengthen observability and deployment readiness. Our goal is to make rfc-rag-agent a reliable, testable engineering assistant rather than a one-off “chat with documents” demo.

Built With

• agent
• ai
• api
• calling
• compose
• computer
• docker
• fastapi
• llm
• multimodal
• ocr
• openai
• postgresql
• pydantic
• pytest
• python
• rag
• react
• redis
• search
• tool
• uvicorn
• vector
• vision