-
-
FastFind: ask a question, get a short answer from NVIDIA Nemotron with citations to the arXiv papers it used.
Inspiration
I'm a master's student at UW Bothell, and my thesis tool, EEGTOOL, turns raw EEG brain recordings into motion predictions. It cleans the signal, extracts features, and compares three models side by side: ridge regression, a dual-stream CNN, and a Transformer. The next goal is to make that pipeline fast enough to run in real time.
That goal sent me into research papers on GPU performance, communication overhead, memory bandwidth, and distributed systems. And that is where I got stuck. Research papers are written for people who already know the exact words to search for. I would spend an evening on keyword searches and still not know whether I had found the right paper, or whether it even said what I hoped it said.
I had never built a search system before. But I kept thinking: what if I could just ask a plain question, like "What techniques reduce NCCL communication overhead?", and get a short answer I could actually check, with the papers it came from? FastFind is the tool I wished I had while working on my own thesis.
What it does
FastFind answers questions over 8,397 arXiv abstracts (distributed systems, performance, hardware, operating systems). You type a question in plain English and get a short answer, written by NVIDIA Nemotron, where every claim carries a citation like [1]. The sources are shown right next to the answer, with the paper ID and its text, so you can check the claim yourself instead of trusting the model.
The part I care about most is what it does when it does NOT know. Ask it "What is the capital of Australia?" and it says it could not find that in the indexed documents. It makes no model call at all in that case, so it cannot make something up, and it saves cost. A research tool that admits "I don't know" is more useful than one that is always confident.
Note: the free host sleeps after 15 minutes, so the first load can take about a minute.
How I built it
- Embeddings: Qwen3-Embedding-8B on Nebius Token Factory, one vector per text chunk (11,297 chunks).
- Search: exact brute-force similarity (one matrix multiply, NumPy on CPU; CUDA through PyTorch is supported when a GPU is present), combined with BM25 keyword search through Reciprocal Rank Fusion.
- Relevance gate: a score cutoff calibrated on my own evaluation set rejects off-topic questions before any model call.
- Answers: NVIDIA Nemotron 3 Nano (30B-A3B) on Nebius Token Factory, told to answer only from the retrieved sources and cite them.
- App: a FastAPI backend and a simple web page that shows where the time went. It has rate limits and a daily cap to protect the API budget, and it is hosted on Render.
One Token Factory key and one endpoint served both the embedding model and Nemotron, which let me spend my time on search quality instead of infrastructure.
Measured results
I did not want to just say it works, so I measured it. I made 100 questions that Nemotron wrote from random abstracts (synthetic, so treat them as a rough guide, about plus or minus 6 points):
- Embeddings alone: 77% recall@1.
- Hybrid search (embeddings plus keywords): 89% recall@1 and 96% recall@5.
- Cutting vectors from 4096 to 1024 numbers makes the index 4x smaller and search about 3.7x faster, for a 1-point drop in recall@1.
- The cutoff rejects all 10 clearly off-topic test questions.
Challenges
Near-miss questions are the hard case: an adjacent topic that the papers only partly cover can still produce an answer built from loosely related sources. I measured this, documented it in the README, and kept the sources visible so a reader can check. The hosted demo runs search on CPU, and I did not run a GPU benchmark for this submission, so I make no GPU speed claims.
What I learned
Retrieval quality mattered more than the model. Adding keyword search gave the biggest gain, bigger than anything I tried with the language model. I also learned that honest evaluation is part of the work: synthetic questions, CPU-only hosting, and no GPU benchmark are all limits I wrote down instead of hiding.
What's next
Benchmark the exact search on GPUs at larger scale, add a reranker, evaluate on human-written questions, and extend the paper collection toward the topics my thesis needs.
Log in or sign up for Devpost to join the conversation.