Watcher

Watcher helps you track updates on websites. It aims to resurrect Google Reader and generate RSS feeds for a user based on online sites and what the user specifies should be watched on the site.

Comment

Inspiration

Google Reader was one of the most beloved tools on the web: simple, fast, and incredibly efficient at keeping people updated. But as the web shifted to SPAs and dynamic content, most of it without RSS, Reader’s death left a real gap.

Watcher was born from the idea: What if we resurrected Google Reader, but upgraded it to haunt the modern web? Meaning:

  • It can watch any page, including SPAs
  • It understands natural language
  • It detects meaningful changes
  • And it exposes everything again as RSS, just like the old days

That mix of nostalgia + modern constraints was the spark.

What it does

Watcher resurrects the Google Reader experience for the modern web.

It lets users:

  • Define a haunt by giving a URL + natural language description like: “Tell me when the admissions page says applications are open for 2026.”
  • Behind the scenes, an LLM generates selectors, keys, and normalization rules.
  • A headless browser (Playwright) scrapes the target on a schedule.
  • Watcher tracks key/value state diffs, not raw HTML, and generates structured change events.
  • Each haunt produces an RSS feed.
  • The UI is a faithful rebirth of the 3-column Google Reader layout, complete with folders, unread counts, stars, refresh, and keyboard shortcuts.

In short: Watcher turns any webpage, even SPAs, into a live RSS source.

How we built it

Watcher is built as a Django-based system with:

  • Spec-driven functional requirements covering scraping, diffing, RSS construction, and a Reader-style UI.
  • Playwright for SPA rendering and key extraction.
  • Celery for periodic haunting and change detection.
  • A fully modeled haunt configuration, derived via LLM from natural language.
  • Structured state tracking, storing only key/value diffs and summaries.
  • RSS feed generation for both private and public haunts.
  • A Google Reader–inspired front-end, implemented to feel as close as possible to the original.

Kiro powered the development loop, particularly around specs, architecture constraints, steering for UI generation, and consistency between backend and frontend layers.

Challenges we ran into

1. SPA scraping complexity

Dynamic sites don’t expose stable selectors. Delayed rendering, XHR calls, hydration timing, all required careful orchestration and a clean abstraction layer.

2. No raw content storage

Watcher intentionally avoids storing HTML for privacy, cost, and security reasons. This demanded a tight spec around normalized key-based state diffs.

3. Mapping natural language → scraping config

The heart of the project: turning a user sentence into selectors, truthy values, and rules. This required iterative spec refinement and steering docs to keep the LLM output deterministic.

4. Recreating the Google Reader UI

Matching fonts, colors, spacing, and layout while using modern tech was surprisingly non-trivial.

5. Ensuring predictable LLM behavior

The system cannot hallucinate. Specs + steering were essential.

Accomplishments that we're proud of

  • Built a modern, AI-powered resurrection of a discontinued but beloved product.
  • Achieved reliable SPA scraping without storing unbounded content.
  • Designed a clean architecture where natural language becomes structured config.
  • Produced authentic-feeling Google Reader UI recreations.
  • Enabled public, shareable RSS feeds for any tracked change.
  • Delivered end-to-end automation from scraping → diffing → alerting → RSS → UI.

What we learned

  • LLMs work exceptionally well when guided by tight specs + steering documents.
  • The web’s move to SPAs made RSS impossible, but not undetectable.
  • State diffs matter more than raw HTML when building meaningful alerts.
  • Nostalgia is a powerful design force, porting old UX patterns into modern stacks teaches discipline.
  • Combining AI + scraping + RSS can create genuinely new value.

What's next for Watcher

  • Multi-key composite change detection (e.g., track lists instead of single fields).
  • Webhooks and API callbacks for automation workflows.
  • Community gallery of public haunts (shared watchlists).
  • AI-based auto-categorization of haunts (Jobs, Admissions, Fellowships, etc.).
  • Mobile-friendly Reader interface.
  • Browser extension for 1-click “Watch this page” haunt creation.

Built With

Share this project:

Updates