Comment Relay

comment made from comment relay
creator studio
reply comments with correct context
comment batching

Inspiration

Creators often spend a surprising amount of time answering the same kinds of questions under their videos—installation errors, setup issues, configuration problems, and requests for clarification.

Each comment may only take a few minutes to handle, but together they create a repetitive support workload that pulls creators away from what they actually want to spend time on: creating.

We built Comment Relay to reduce that workload without giving an AI unrestricted access to a creator’s public audience.

What it does

Comment Relay is a creator-controlled decision inbox for YouTube comments.

A creator connects their YouTube channel, syncs their videos and recent comments, and runs the Relay Agent. From there, the agent helps organize the work by:

Grouping comments that share a similar intent.
Prioritizing conversations that need attention first.
Creating decision packs for each recurring type of question.
Drafting context-aware replies.
Showing the confidence, evidence, and reasoning behind each grouping.
Holding back ambiguous, sensitive, or low-confidence responses for human review.

The creator stays in control throughout the process. They choose which comments to respond to, edit the draft if necessary, and explicitly approve the final response before it is published through the YouTube API.

The agent never posts replies autonomously.

Comment Relay also includes Script Studio, a separate workspace for turning repeated audience questions into future content.

If the same topic keeps appearing in the comments, the creator can turn it into a video idea and develop it through an editable workflow:

Idea → Brief → Outline → Script → Packaging

This means one recurring question can become more than just another support task—it can become the starting point for the creator’s next useful video.

How we built it

The frontend is built with React and Vite, using a lightweight dashboard interface inspired by modern creator studios.

The backend uses Python and FastAPI, with integration to the YouTube Data API v3 through Google OAuth 2.0 with PKCE.

For persistence, we use SQLAlchemy, with SQLite during local development and PostgreSQL-compatible configuration for deployment.

The agent workflows are built using the Strands Agents SDK with structured Pydantic outputs.

The Reply Desk agent uses tools to:

Read creator-approved context such as FAQs, known fixes, preferred voice, and safety rules.
Enforce the human approval boundary whenever a reply is uncertain, sensitive, or requires creator judgment.

The Script Studio agent uses tools to:

Read the creator’s brief, audience, and content constraints.
Flag claims in generated scripts that should be verified before publishing.

The model provider is configurable. Amazon Bedrock is the default AWS-compatible option, while direct Anthropic support is available for local development.

If live model credentials are unavailable, the application falls back to a transparent, deterministic workflow so the complete product experience can still be demonstrated.

Important engineering decisions

One of our main design decisions was to keep the agent deliberately bounded.

The system can organize comments, identify patterns, prepare responses, and surface recommendations—but the creator remains responsible for every public action.

We also designed synchronization as a reconciliation process rather than simple database upserting.

For example, if a comment is deleted from YouTube, the next synchronization removes it from the local database. If that deletion leaves a decision pack with no active comments, the pack is removed as well.

This prevents deleted or outdated conversations from resurfacing in future agent runs.

We also kept Reply Desk and Script Studio intentionally separate.

Reply Desk focuses on the core problem: grouping comments and helping creators respond safely with human approval.

Script Studio is an independent content-planning workflow that turns recurring audience needs into potential future videos.

Challenges

One of the biggest challenges was implementing Google OAuth correctly while handling YouTube permissions and keeping the local database consistent with the remote YouTube state.

The OAuth flow required careful PKCE state management so the verifier created during authorization was still available when the callback occurred.

We also had to configure the correct YouTube permissions and account for Google’s restrictions when an OAuth application is running in testing mode.

Another challenge involved preventing duplicate decision packs during synchronization.

Several comments may belong to the same group before the database transaction has been flushed. If that is not handled carefully, the system can accidentally create multiple packs for what should be a single conversation group.

To solve this, we track pack IDs inside the synchronization operation before creating any new records.

The final challenge was defining the right agent boundary.

We wanted the system to be useful enough to produce practical, relevant replies without becoming so autonomous that it could speak publicly on the creator’s behalf without oversight.

As a result, the agent intentionally stops when a question is ambiguous, sensitive, low-confidence, or clearly requires the creator’s judgment.

What we learned

The biggest lesson from building Comment Relay was that the most useful role for an agent in a creator workflow is not to replace the creator.

Instead, the agent should handle the repetitive discovery and preparation work, then bring the important decisions back to the human.

We also found that grouping similar comments is much more valuable than simply generating replies one comment at a time.

A group of repeated questions tells the creator something important about their audience. It reveals where people are confused, what they care about, and what information may be missing from the existing content.

Those same patterns can then become ideas for future videos.

Future work

There are several directions we would like to explore next.

We plan to integrate Exa for topic discovery and current web signals, and Firecrawl for extracting useful information from external sources.

We also want to improve creator-context management, build evaluation datasets for measuring grouping quality, strengthen token storage and authentication security, and support scheduled agent execution through an AWS worker or AgentCore.

Over time, these improvements could make Comment Relay more useful while preserving the design principle we started with:

Let the agent handle the repetitive work, but keep the creator responsible for the final public voice.