Inspiration
I like researching things on my own often, through my own preferred sources. But I not only spend a lot of time sifting through tabs before finding what matters, but also regularly lose interest and switch to something distracting. And by the time the browsing session is "done," there's no good way to ask "what did I just spend the last hour reading?" Existing browser history is just a list of URLs. It doesn't capture what was read or why it mattered. I wanted browsing history to become something query-able, like a personal research log you could actually have a conversation with. And go beyond with more tools like suggesting further reads/YT vids, periodically send my browsing sessions as compact Email digests, etc.
What it does
Browsing Knowledge is a Chrome extension that tracks your browsing session (pages visited, time spent on each, and the full article text) and indexes it into Elasticsearch with semantic embeddings. When you end a session, an analysis page opens showing time-per-domain charts and your navigation history, plus a chat panel powered by a Gemini-based agent. You can ask the agent things like "what did I learn about X?" or "how did I spend my time, and what distracted me?" It answers using your session's data. You can also ask for further reading and YouTube videos; the agent grounds its suggestions in real search results (no hallucinated links) and a one-click button opens all suggested pages as new tabs. There's also an option to email yourself a digest of the session with key insights and further reading.
How I built it
The extension (Manifest V3, esbuild) tracks tab focus/blur events and uses Readability.js to extract clean article text, posting visit records tagged with a session ID to an Elasticsearch Serverless deployment. ELSER provides semantic embeddings on the page content automatically. On the agent side, we used Google's Agent Development Kit (ADK) with Gemini 2.5 Flash. Two ES|QL tools (search_session and focus_analysis) are defined in Elastic Agent Builder and exposed over its built-in MCP server. The ADK agent connects to this MCP server via McpToolset, so retrieval and aggregation both happen inside Elasticsearch with a hard session_id filter baked into the query itself. A second agent handles further-reading suggestions using ADK's grounded Google Search tool, and we extract real URLs from the grounding metadata to populate the "open suggested pages" action. A small FastAPI server exposes /chat and /digest endpoints that the analysis page calls, with the digest sent via the Gmail API. It was my first time exploring all these tools! So I ran into quite a few stumbling blocks, but had the help of the in-built chatbot of Elastic and also the official docs.
Challenges I ran into
Getting the MCP connection working took a while. Early on, an index_search-type tool only treated session_id as an LLM-level "suggestion" rather than an enforced filter, so the agent could (and did) leak content from other sessions. Rebuilding search_session as an ES|QL tool with a hard WHERE session_id == clause fixed this properly. We also hit a wall combining Gemini's built-in google_search tool with custom function-calling tools in the same agent, Gemini's API rejects that combination outright, which led us to split into two agents and route between them based on query intent. Separately, Gemini kept fabricating YouTube links; the fix was extracting real URLs from grounding metadata rather than trusting the model's prose. And on the infrastructure side, we burned a fair amount of time navigating Gemini API free-tier quota limits across multiple Google Cloud projects before landing on a stable Tier 1 setup.
Accomplishments that I'm proud of
Getting the agent to be genuinely agentic rather than just RAG over, say, a vector-store. It retrieves, analyses, searches the web, and then acts by opening tabs on the user's behalf. We're also proud of the strict session-scoping guarantee: the agent architecturally cannot answer with another session's data, which matters a lot for a tool handling personal browsing history. And going from "a rough extension that dumps JSON" to a working Elastic + ADK + Gemini pipeline with a polished UI in the space of this hackathon was quite an exciting and fun challenge in the end ;)
What I learned
A lot about the practical seams between Elastic Agent Builder, MCP, and Google's ADK. Particularly that "the LLM was told to filter by X" and "the query is hard-filtered by X" are very different guarantees, and for anything privacy-sensitive only the latter is acceptable. We also learned that grounding metadata is more trustworthy than a model's generated text when real URLs matter, and that Gemini's tool-combination restrictions (built-in vs. function-calling tools) need to be designed around early, not discovered late ;)
What's next for Browsing Knowledge
LOCAL! I want to make a local-first version, but allow for cloud-based inference option with selective history tracking (maybe use an AI model itself to determine what pages are potentially not private to track). Full future scope - Periodic, cross-session digests (e.g. "summarise everything I researched this month") rather than only per-session summaries; an AI-assisted denylist so sensitive pages (banking, health, private messages) are never indexed in the first place; a fully local mode using a self-hosted Elasticsearch and Ollama for users who want zero cloud calls; more efficient capture that only indexes tabs actually visited rather than all open tabs; and eventually a team mode where a shared index can surface collective research and highlight knowledge gaps across a group.
Log in or sign up for Devpost to join the conversation.