Sift

Inspiration

I read a lot of articles on the web and I wanted an assistant that can help me deep dive into particular topics. It shouldn't just be constrained to the article I am on, but it should have the capability to relate with other articles that discuss similar topics.

What it does

Sift is a Chrome side panel. Open any article, ask a question, and it answers from the page. If the page doesn't cover the question, it searches the rest of the site through Tavily and tells you so. Every answer carries a label: from the page, from the web, from both, or a grey not checked. Quotes are clickable and highlight the passage in the page. A thread can also be shared as a link that opens read-only in the panel.

How I built it

• Model: NVIDIA Nemotron 3 Nano through Nebius Token Factory, with Super and Ultra tested. Nano is the default.
• Agent loop: the model decides when to call a tool: search_site (Tavily, with the domain pinned in code so only the query comes from the model), fetch_page, search_page and cite_page. Three tool rounds at most, then a forced answer.
• Verified quotes: to answer from the page, the model calls cite_page with quotes. The service worker checks each against the page text and only then assigns the from the page label. The label comes from what the code observed, never from what the model claims.
• Long pages: when a page is too long for the prompt, the full text stays in the browser and the model searches it locally with search_page. The same applies to a fetched page that is too long.
• Stack: Manifest V3, React 19, Vite, Readability for extraction, and a small proxy so judges can try it without their own keys.
• Regression net: a live eval set runs the real agent loop against a fictional-site fixture and real long pages, so a prompt change can't quietly bring back ungrounded answers.

Challenges I ran into

Making sure the labels could be trusted. My first version had the model say whether the page answered the question. When I moved to a loop where the model picks its own tools, that guard disappeared, and nothing stopped it from answering out of memory. So I built an eval set that runs the real agent loop on real pages. On 45 answers, none of the 32 labelled from the page lacked real support, and none of the 45 were confident fabrications. 13 came back not checked instead of claiming a label they hadn't earned.

Checking quotes comes at a price. The model has to pick what to quote before it answers, and that extra round made the same question take about 1.5x to 4x longer than without it. I tried tuning the prompt and it barely moved. In the end I kept the check, because a slower answer I can trust beats a fast one I can't.

Some problems from the API:

• Stalled streams. In my eval, a Nebius stream once hung for 900+ seconds because nothing in my client had a timeout. I fixed it by aborting after 30 seconds of silence, so a long answer isn't cut off.
• Ending the loop. I wanted a hard stop after three tool rounds. tool_choice: "none" didn't work: the model kept asking for another search, and the request came back as raw tool-call text in 5 of 5 runs. It worked after removing the tools and adding an "answer now" message (5 of 5 clean).
• Rate limits. When I ran two evals on one key, 17 of 60 runs failed with a 4xx, and my client told me my API key was wrong, due to rate limiting.

What I learned

• Measure the model you actually ship. Nano was faster, cheaper, and used tools more sparingly than Super here, and it was the only one of the three with prefix caching on my account. Ultra passed 4 of 6 runs of the grounding gate against Nano's 3 of 3, probably because my prompt was tuned on Nano.
• Let code decide what is true, and let the model decide what to look at.
• Compare latency only within the same window. The same endpoint varied about 5x between sessions.

What's next for Sift

More real pages in the grounding eval, a measured latency breakdown for simple questions, and a version of the store listing with narrower host permissions.

Built With

• manifestv3
• nemotron
• tavily