Description

Add a Twitch connector to packages/connector/twitch that turns followed channels, recent VODs, and clips into documents through one cognee.remember call. Documents use stable Twitch IDs and per-channel node sets, with incremental DLT state and hard-delete rows for unfollows and confirmed missing VODs/clips.

The connector validates the user token with /helix/users, paginates follows and content, retries rate limits/server errors with bounded waits, and honors remaining quota/reset headers. All API reads complete before rows are published; failed or interrupted extraction cannot advance state or emit speculative deletes. Account switches under an existing resource are rejected, and credentials are not persisted.

Helix has no VOD timestamp filter, so VOD discovery walks newest-first to the saved watermark; clips use explicit time windows with subdivision before the API's approximate 1,000-result ceiling. Previously stored IDs are verified separately, so falling outside the initial lookback is never mistaken for deletion. A five-minute overlap covers near-boundary content; older backdated publications can be missed. The README explains API field limitations, token setup, retention, and a runnable remember/recall example. Includes a path-scoped Python 3.10/3.13 CI workflow.

Fixes topoteretes/cognee#6489

Testing

Validated locally on Python 3.13.10, Cognee 1.6.6 (core main), DLT 1.30.0, and HTTPX 0.28.1:

  • TWITCH_TEST_LADYBUG_HOME=/tmp/cognee-twitch-ladybug .venv/bin/pytest packages/connector/twitch/tests -q --tb=short: 44 passed.
  • Fake HTTP tests cover pagination/repeated cursors, incremental no-op/new content, metadata and login changes, unfollows, expired VODs (including all-missing HTTP 404), clips, account switching, auth failures, retry recovery/exhaustion, quota headers, wait limits, cancellation, token redaction, and VOD duration formats.
  • Real DLT + SQLite checks persisted state, merge, and deletion of the final row.
  • Real Cognee remember/recall tests use SQLite, Ladybug, and LanceDB with fake LLM/embedding calls. They verify searchable documents, repeat-sync identity, VOD expiry, and physical graph/vector cleanup after the final unfollow.
  • ruff check and ruff format --check on the package: passed.
  • git diff --cached --check: passed before committing.
  • Wheel and source distribution build: passed; wheel installed and imported from /tmp outside the checkout.

No live Twitch credentials or paid model calls were used. Ladybug's native JSON extension was installed separately in /tmp before offline integration testing.

DCO Affirmation

I affirm that all code in every commit of this pull request conforms to the terms of the Topoteretes Developer Certificate of Origin.

Built With

Share this project:

Updates

Submission history