Inspiration
The starting point for Abyss Echo was a recurring moment after watching a video: I had an immediate reaction and wanted to exchange it with someone who had actually watched the same thing and felt something close enough to continue the thought.
The existing options all introduced friction. A dedicated conversation with a friend felt too formal, and that friend might not have watched the whole video. I could post a comment, but it might receive no reply. I could search through a long comment section, but finding the right response took more effort than the brief, spontaneous moment justified. I could talk to a language model, but a single model could not replace the variety of real human perspectives I wanted to encounter.
That led to the core question: can a comment system shorten the path between having a reaction, meeting a relevant real human expression, and continuing a natural exchange?
My original concept was not a chatbot and not another recommendation feed. It was a three-layer interaction structure in which the video, its characters and entities, observable events and timeline form a sparse shared graph; real comments form a second, softly connected graph around that structure; and newly confirmed human expressions can eventually become part of a growing interaction network for future viewers.
I later selected Blender Studio’s Sprite Fright only as the demonstration medium because a high-quality open movie could be shown with clear attribution under CC BY 4.0. The film was not the source of the product idea.
What it does
Abyss Echo is a comment-interaction capability for video and social platforms.
It uses a real video and real human comments to help a viewer meet a nearby human perspective while their own reaction is still fresh. A prebuilt sparse video graph and local normalization help interpret known names, aliases, and approximate wording. Graph retrieval finds related human sources, and GPT organizes one selected source into a more conversational response while preserving meaning and provenance.
The Build Week website is a local Windows reference implementation of that engine. A viewer can watch Sprite Fright, speak a reaction, inspect and correct the local STT transcript, and explicitly confirm the turn. The system then:
- searches a processed graph of 2,023 anonymized comments and replies;
- applies quality, safety, and stable-deduplication gates, leaving 1,579 records eligible for live matching;
- combines semantic proximity, fuzzy matching, aliases, video anchors, native reply edges, and comment-graph neighborhoods;
- sends the twelve strongest unused candidates to one constrained GPT-5.6 Sol call through Codex;
- receives one allowlisted source ID and a fresh, source-bound conversational adaptation; and
- reads the disclosed adaptation through local Kokoro TTS.
The viewer can immediately respond again. Continuous viewer turns are never added to the response pool. A temporary session ID and used-source list keep each test separate and prevent a previously used source from replying again.
The interface presents the result through three connected views:
- Dialogue Weave is the live STT → retrieval → AI adaptation → TTS exchange.
- Source Archive exposes the immutable original text, stable ID, source, parent/reply relationship, matched video anchors, and processing disclosure.
- Resonance Echoes shows other nearby comments and offers original/adaptation views. When no reviewed adaptation exists, the interface says so and preserves the original instead of fabricating one.
The current release supports complete English and Simplified Chinese interface and speech paths. It also includes a small, reversible Connection Safety prototype for adjusting interaction links when a continuing exchange escalates, without deleting comments or deciding which person is right.
Because I remained concerned about downstream data authorization and privacy, I did not publish the processed comment pool as a public dataset. The complete pool is available only in the private judge repository; the public demo shows only the few source records selected during the recorded interaction.
The private repository contains the processed, pseudonymized, limited-retention pool needed to reproduce retrieval, while the raw export, author identities, identity mapping, and generated audio remain excluded. Viewer turns are transient and are not posted back to YouTube.
How I built it
I began with the product structure: a sparse video graph centered on people and entities, observable events, explicit facts and dialogue, sequence, and a whole-video fallback; a comment graph with weighted soft connections rather than forced single labels; and three interaction branches for fast discovery, conversation, and provenance.
Codex running GPT-5.6 Sol was my primary engineering collaborator. I used it to turn those constraints into the FastAPI architecture, Pydantic schemas, provider boundaries, data-import pipeline, retrieval and graph rules, bilingual interaction flow, Windows launcher, release checks, and 111 automated tests.
The primary /feedback Session ID is:
019f845a-90cf-75f2-827d-c131a72d135e
The backend is built with FastAPI, Pydantic, and SQLite. The interface uses plain HTML, CSS, and JavaScript served from the same local process.
A reproducible import pipeline pseudonymizes the source data, removes inline handles, creates stable IDs, preserves 1,723 native parent/reply edges, connects 371 comments to specific video entities or events, and builds 2,240 bounded semantic edges.
Low-confidence records may remain connected only to the whole film, to comment neighbors, or not at all. The graph is not completed with invented precision.
Speech remains local. SenseVoiceSmall produces the editable English or Chinese transcript. Kokoro-82M runs as a persistent worker, prewarming in the background while the browser opens and speaking with an English or Chinese generic synthetic voice.
STT, TTS, and language-model functions are replaceable providers so a platform or judge can substitute stronger models without changing the interaction contract.
I initially explored a lightweight local 4B language model because I wanted a deployable, low-cost path. Testing showed that it could merge the viewer’s wording or identity into another person’s comment, or become so constrained that it merely copied the source. That failed the project’s quality requirement.
The final default therefore uses one authenticated Codex call backed by GPT-5.6 Sol to select and adapt, without requiring a separately billed OpenAI API key for the demonstration.
The selected source is the required content authority by contract. Local guards reject an unlisted ID, the wrong language, source passthrough, and detectable substantial viewer-only wording.
Challenges I ran into
The central challenge was not simply semantic search.
The response had to be close enough to sustain conversation without becoming identical repetition, and natural enough to feel like a reply without inventing a new person’s opinion. Source-only retrieval felt mechanical; unconstrained generation damaged identity and meaning.
The working compromise combines full-pool graph retrieval, a strict candidate allowlist, one-source adaptation, immutable provenance, and local output guards.
The second challenge was model quality under real resource constraints. I had no separate API budget and originally expected a small local model to be sufficient. Measured failures showed that keeping an unsuitable model would create the appearance of local efficiency while lowering the actual product quality.
I kept lightweight local models where they worked well—STT and TTS—and moved the meaning-sensitive selection and rewriting step to GPT-5.6 Sol through Codex.
Latency also mattered. A high-assurance prototype used separate model calls for selection, adaptation, and verification, but it was too slow for the primary demo. I retained that route as an optional mode and created a one-call fast path.
Kokoro initialization moved to a persistent background worker so the browser opens immediately while speech prewarming continues.
Finally, the release had to balance reproducible judge testing with my concern about comment-data authorization. That shaped the private-repository decision, pseudonymization, 30-day retention metadata, separation of originals and AI adaptations, and exclusion of raw identity-bearing files.
What I learned
I learned that the useful distance between two comments is not the same as simple agreement.
An identical reaction often adds little. A nearby perspective, explanation, gentle difference, or extension can create a better exchange. That is why the product uses weighted soft relationships rather than forcing every comment into one topic or emotion category.
I also learned that provenance cannot be an invisible engineering log. If AI makes a human comment more conversational, the original source, processing boundary, and relationship information must remain part of the interface.
Prompts alone are not a sufficient safeguard. Typed schemas, candidate allowlists, stable IDs, signed text/source/language bindings, local validation, and visible fallback states all contribute to fidelity.
The model evaluation also changed my engineering assumptions. Small local models are valuable when the task and quality ceiling fit, but using one merely to claim a local stack is inefficient if its errors undermine the product.
A replaceable provider architecture made it possible to keep speech local, use a stronger model for the meaning-sensitive step, and leave a clear upgrade path.
Accomplishments that I am proud of
- A working two-turn voice loop: local STT → full-pool graph retrieval → one traceable GPT-adapted source → local TTS → another real viewer turn.
- A processed 2,023-record comment graph with 1,579 live-eligible sources, 1,723 native reply edges, and 2,240 semantic edges rather than eight fixed responses.
- A source-binding boundary that rejects unlisted sources and substantial viewer-only wording, reducing the risk that the viewer’s message is presented as someone else’s perspective.
- A provenance interface that keeps the human original and AI adaptation visibly separate.
- Coupled English and Simplified Chinese interface, text, STT expectation, and TTS.
- A replaceable provider architecture, Windows one-click launcher, graceful degradation paths, 111 automated tests, and a submission preflight.
What’s next
The competition website is a reference implementation, not the final product boundary.
The next step is to package the retrieval, graph, source-binding, and UI contracts as an embeddable SDK/API and component set for video platforms, social networks, and content communities.
For a production deployment, I would work with a platform that owns the comment relationship or build an explicit opt-in contributor pool, rather than depend on a privately reviewed research cache.
I would add authenticated multi-user sessions, consent and deletion controls, persistent vector and graph indexes, stronger multilingual speech providers, formal source-fidelity evaluation, and user studies of whether selected nearby responses actually sustain meaningful interaction.
I also want to develop the third layer from the original concept: confirmed new human expressions entering the network, accumulating authentic reply relationships over time, and becoming useful context for future viewers—without automatically posting them back to an external platform.
The lightweight Connection Safety prototype can then evolve into a trajectory-aware, symmetric, reversible link-control module rather than a system that judges or deletes speech.
Log in or sign up for Devpost to join the conversation.