LiveReply Copilot

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for LiveReply Copilot

Inspiration

Livestreamers have to look at the camera, speak, moderate the room, and scan a fast-moving comment feed at the same time. Missing a thoughtful question or a high-intent viewer can mean a lost conversation, follow, gift, or sale. We wanted to build a “second brain” that reduces the cognitive load without taking the creator’s voice or control away.

What it does

LiveReply Copilot is a Windows desktop companion for Chinese livestreamers. It reads the publicly visible interaction-message area of a livestream companion window through a local screenshot and OCR pipeline, then sends the normalized comment and the streamer’s profile to a DeepSeek-compatible API. It returns three short, speakable reply options:

• warm / lightly flirty
• funny / playful
• room-control / boundary-setting

Each suggestion includes a simple intent or safety cue. The streamer can choose, copy/read, skip, or regenerate a reply. The app never automatically posts comments, logs in to Douyin, or acts on the streamer’s account.

How we built it

The app uses Electron and Node.js with a local HTTP server. Screenshot capture, image preprocessing, and Tesseract.js OCR turn the visible comment panel into text. A comment normalizer removes OCR noise and keeps the prompt compact enough for low-latency generation. The DeepSeek-compatible chat API produces the three response styles under a short spoken-language constraint.

During the event period, we focused on making the prototype reproducible and safer to evaluate: an English onboarding/testing flow, a synthetic-comment demo mode, feedback-based reply ranking, and a local-first privacy boundary. Adopt, skip, and regenerate feedback becomes a lightweight ranking signal for future suggestions, while the creator can always override the recommendation.

Challenges

The hardest part is not generating a clever sentence. It is making the sentence useful in a live room. OCR has to cope with a moving vertical phone screen, small fonts, avatars, and changing overlays. The model must stay concise, respond to the actual conversational intent, and avoid sexual, abusive, or boundary-crossing escalation. We addressed this with image preprocessing, text normalization, explicit persona and forbidden-topic settings, multiple reply styles, and a human approval step.

Why this approach matters

This is an assistant, not an auto-reply bot. The creator remains responsible for what is said and when it is said. The local OCR fallback is useful when an official live-comment API is unavailable, while the design avoids collecting private viewer data. Demo comments should be synthetic or used with permission, and the project must follow the livestream platform’s rules and applicable privacy requirements.

What’s next

We plan to publish a reproducible test build and public source repository, measure OCR accuracy and end-to-end latency on representative screen layouts, and add stronger multilingual support. If an official livestream-comment API becomes available, it can be evaluated as an opt-in input alongside the local OCR path.

Built With

• sharp