Inspiration
Reading ML papers one at a time is slow, and the interesting questions span several of them: LoRA vs. adapters, InstructGPT vs. earlier RLHF work, Switch Transformer vs. the original sparsely-gated MoE layer. When I tried plain RAG on a question like "compare LoRA with the adapter methods in the PEFT survey", it answered almost entirely from the LoRA paper, because "LoRA" dominated the search query. I wanted an assistant that notices when a question needs several papers and checks that it actually used them.
What it does
Technical Knowledge Assistant answers questions across 11 foundational ML papers (Transformers, RoPE, MoE, Switch Transformer, LLaMA, LoRA, a PEFT survey, three RLHF papers, and RAG). It shows a live view of the agent's steps, cites the passages behind every answer, and returns comparisons as tables. It handles follow-up questions, and it politely declines questions outside its knowledge base instead of making something up.
How I built it
- Ingestion: PDFs are parsed to markdown, split at section headings into ~450-word chunks (497 in total), with references and appendix noise removed. Each chunk keeps its paper and section as metadata.
- Retrieval: chunks are embedded with Qwen3-Embedding-8B through Nebius Token Factory and stored in Chroma.
- Agent (LangGraph): a state machine with a scope check and query planning, retrieval, a coverage check that rewrites the queries and retries once if a paper the question names is missing, and a final answer step.
- NVIDIA models on Token Factory: Nemotron-3.5-Lightning handles the fast, frequent planning and grading calls, and Nemotron-3-Ultra writes the final cited answer, so the large model runs once per question.
- UI: Streamlit, with chat history, a status box for the agent's steps, sources, and the agent trace. It is deployed on Streamlit Community Cloud.
Indexing all 11 papers cost a fraction of a cent.
Challenges
- A reasoning model that ate its own token budget. Lightning wrote its thinking into the reply, so my small token limit was used up before any JSON appeared, and my parser silently fell back to the raw question. It looked like a retrieval bug for a while. The fix was disabling thinking for the short planning calls, raising the budget, and making the parser tolerant. Silent fallbacks hide bugs, so I now log them.
- One paper dominating the results. Decomposing the question into one query per paper, plus a grader that rejects results missing a named paper, fixed the LoRA-vs-PEFT case.
- Follow-ups and off-topic questions. The agent rewrites follow-ups into standalone questions using chat history, and an LLM scope check routes unrelated questions to a decline.
- Platform friction: the API model ID differed from the catalog name, and no NVIDIA embedding model was available on the public endpoints, so retrieval uses a Qwen embedding model.
What I learned
Agent loops need visible failure modes, model routing matters for cost and latency, and good chunking and metadata matter as much as the model.
What's next
An evaluation harness that scores retrieval and answer grounding, hybrid keyword and vector search with a reranker, and a citation-verification step that checks every cited claim against its passage.
Built With
- chromadb
- langgraph
- nebius-token-factory
- nemotron-3-ultra
- nemotron-3.5-lightning
- nvidia-nemotron
- pymupdf
- python
- qwen3-embedding
- rag
- streamlit
