Description
Add a Twitch connector to packages/connector/twitch that turns followed channels,
recent VODs, and clips into documents through one cognee.remember call. Documents
use stable Twitch IDs and per-channel node sets, with incremental DLT state and
hard-delete rows for unfollows and confirmed missing VODs/clips.
The connector validates the user token with /helix/users, paginates follows and
content, retries rate limits/server errors with bounded waits, and honors remaining
quota/reset headers. All API reads complete before rows are published; failed or
interrupted extraction cannot advance state or emit speculative deletes. Account
switches under an existing resource are rejected, and credentials are not persisted.
Helix has no VOD timestamp filter, so VOD discovery walks newest-first to the saved watermark; clips use explicit time windows with subdivision before the API's approximate 1,000-result ceiling. Previously stored IDs are verified separately, so falling outside the initial lookback is never mistaken for deletion. A five-minute overlap covers near-boundary content; older backdated publications can be missed. The README explains API field limitations, token setup, retention, and a runnable remember/recall example. Includes a path-scoped Python 3.10/3.13 CI workflow.
Fixes topoteretes/cognee#6489
Testing
Validated locally on Python 3.13.10, Cognee 1.6.6 (core main), DLT 1.30.0, and HTTPX 0.28.1:
TWITCH_TEST_LADYBUG_HOME=/tmp/cognee-twitch-ladybug .venv/bin/pytest packages/connector/twitch/tests -q --tb=short: 44 passed.- Fake HTTP tests cover pagination/repeated cursors, incremental no-op/new content, metadata and login changes, unfollows, expired VODs (including all-missing HTTP 404), clips, account switching, auth failures, retry recovery/exhaustion, quota headers, wait limits, cancellation, token redaction, and VOD duration formats.
- Real DLT + SQLite checks persisted state, merge, and deletion of the final row.
- Real Cognee
remember/recalltests use SQLite, Ladybug, and LanceDB with fake LLM/embedding calls. They verify searchable documents, repeat-sync identity, VOD expiry, and physical graph/vector cleanup after the final unfollow. ruff checkandruff format --checkon the package: passed.git diff --cached --check: passed before committing.- Wheel and source distribution build: passed; wheel installed and imported from
/tmpoutside the checkout.
No live Twitch credentials or paid model calls were used. Ladybug's native JSON
extension was installed separately in /tmp before offline integration testing.
DCO Affirmation
I affirm that all code in every commit of this pull request conforms to the terms of the Topoteretes Developer Certificate of Origin.
Log in or sign up for Devpost to join the conversation.