Summary Trace

See what your AI summary is really based on—trace every point back to its source

Comment

Inspiration

The project was inspired by the growing curiosity and skepticism around AI-generated summaries. Users often wonder what sources or parts of a page an AI actually relied on to create its concise takeaways. This tool aims to make AI summarization more transparent and trustworthy by linking each summarized key point back to its original reference on the web page—bridging the gap between machine output and human understanding.

What it does

The extension analyzes AI-generated summaries and maps each key point back to its original source on the web page. It visually highlights or links the summarized content to its reference passages, helping users verify accuracy, explore context, and build trust in AI outputs.

In detail, users can open the extension in a side panel to analyze the current web page. The extension gathers the page content and relevant metadata, then sends it to an API for summarization and key point extraction. When the AI returns the summary, each key point includes reference indices pointing to the original paragraphs. Users can hover over a key point to automatically scroll to and highlight the corresponding section on the page for easy reference.

How we built it

Built as a Chrome Extension powered by Chrome AI APIs:

  • Content script: Extracts page text
  • Prompt API: Detects topics and links between them
  • Solidjs: A high performance UI framework
  • Local storage: Keeps analyzed pages private
    All processing happens locally — no cloud, no tracking.

Challenges we ran into

  • Local API performance issue. Structured Output has to wait until all of the bytes to return before parsing. This adds lots of delay to the end user and making stream prompt useless.
  • We then explored ways to improve the summary experience to help users understand web pages more quickly. However, we discovered that there aren’t many reliable systems for evaluating summary quality. Human evaluation is costly and often inconsistent, while using LLMs as judges lacks transparency and explainability.
  • We hypothesized that providing a specific goal could influence the key points extracted by the model. For example, if a user only wants to find a restaurant’s phone number, there’s no need to summarize the entire introduction. However, our initial results show this is rarely the case, and we plan to continue researching why.

Accomplishments that we're proud of

  • Use Prompt API to achieve the goal.
  • Allow user to interact with the page with a follow up questions
  • Improved API first-response time by over 90% (from 40s to 3s) through session separation, streaming prompts, and pre-warming techniques.
  • Improved observability by logging the AI’s reasoning chain (CoT) to the console, enabling developers to debug and understand key point and reference selection.

What's next

  • We will research a better reference-less summarization eval that focus on summary accuracy and information intensity as oppose to comprehensive.
  • Use a local user knowledge graph to evaluate the importance of a key point. If user already know some information, the related key point should be lower priority compared with other new knowledge.

Built With

Share this project:

Updates