Feedbackai-agent

An autonomous agent for early-stage startups that reads product feedback in French and English, clusters it, and opens real GitHub tickets for the problems that matter, on its own.

Comment

Inspiration

Early-stage SaaS teams drown in feedback. It arrives from everywhere at once: a support inbox, a signup form, app store reviews, sometimes in different languages. A team of two or three people, already stretched across all of it, never has time to read everything, let alone notice that the same problem is being reported ten different ways. That is where the signal that should be driving the roadmap quietly gets lost.

Most feedback tools don't help with the part that actually matters. They collect it and put it in a dashboard, then leave a human to spot the pattern, decide it's worth acting on, and open the ticket. All Things Agentic was the push to build the piece that's missing: an agent that reads the feedback, reasons about it, and acts on it on its own, on an external system a real engineering team already uses, instead of handing all of that back to an already-overloaded person.

What it does

feedbackai-agent takes raw product feedback in French or English, from typed text, a CSV export, or an App Store URL, and turns it into action with no human triaging anything. It scrubs out personal data first, then tags each item with a language, category, sentiment, and a short summary. Similar feedback is grouped by meaning rather than keywords, so the same complaint in two languages becomes one cluster instead of two. When a cluster is both frequent and negative enough, the agent opens a real GitHub issue for it and notifies the team, on a recurring schedule, on its own. You can also ask it directly, in either language, what is breaking this week.

How I built it

Feedback comes in as free text, a CSV upload, or a URL (an App Store page pulls recent reviews through Apple's public RSS feed; any other page gets its main text extracted). Every item is scrubbed of emails and phone numbers, then triaged in one model call: language, category, sentiment, and a one-line summary. Triage runs on Gemma, deployed through Vertex AI Model Garden, chosen specifically because it's cheap enough to run on every single feedback item without worrying about volume; Gemini is the fallback if that endpoint is down.

On a schedule (Cloud Scheduler, through Pub/Sub, hitting an OIDC-verified endpoint), the agent wakes up, re-clusters unclustered feedback by embedding similarity (not keywords, so a French complaint and its English twin land in the same group), generates a short weekly report, and for any cluster that's both frequent enough and negative enough, writes and opens a real GitHub issue, then notifies the team. The whole thing is orchestrated with Google ADK: search, clustering, insights, and ticket creation are tools the agent can call both from that scheduled loop and from a chat interface where you can just ask it a question.

Team-facing output (labels, tickets, notifications) comes out in whichever language the team configured, French or English, while the original feedback quote inside a ticket stays in its own language. That distinction, and getting the language of a feedback item detected deterministically instead of guessed by a model, turned out to matter more than I expected.

Challenges I ran into

The one that ate the most time: Vertex AI's "dedicated endpoint" deployments (the default for Model Garden's one-click Gemma deploy) live on their own domain, not the shared regional one, and that domain doesn't speak gRPC. The SDK's REST transport, when I tried it, added query parameters the endpoint's gateway didn't recognize and rejected outright. I ended up bypassing the SDK for that one call and sending a raw authenticated HTTP request that mirrors exactly the curl example Vertex's own console generates, then discovered the response shape itself was different from what I'd assumed: under this container's chatCompletions request format, predictions isn't a list of instances, it's the chat completion object directly.

The subtlest bug wasn't a crash, it was silent data corruption. Each autonomous cycle re-clusters feedback and needs to know "is this the same cluster as last time," so it doesn't spam duplicate tickets. I'd built that identity check around the cluster's display label, but that label is regenerated by Gemini on every cycle and isn't guaranteed to come out the same way twice for the same group of feedback. Every cycle quietly created a new database record for what a human would call the same cluster, and the very idempotence check meant to stop duplicate tickets never got a chance to fire, because it was always looking at a "new" cluster it had never seen before. Two GitHub issues about the same dashboard slowness complaint sat open before I traced it.

The same class of bug came back during actual filming, in a nastier form. With the fix in place, a single cluster could no longer be double-ticketed by two things happening at once, but I hadn't accounted for Pub/Sub simply retrying a slow delivery: once the pipeline started running its model calls through Vertex AI, cycles occasionally took long enough to outlast the subscription's acknowledgment window, so Pub/Sub redelivered the same trigger a few minutes later. Four duplicate GitHub issues appeared over about forty minutes, each timestamp roughly eleven minutes apart, which is what gave the pattern away. The real fix wasn't just an atomic "claim" on the cluster before doing any work (though that stops true concurrent duplicates); it was accepting that a cycle's own idempotence check has to survive being asked the same question multiple times, arbitrarily far apart.

And a genuinely embarrassing one: the regex that masks emails ran after a text-cleaning step that stripped out characters the masking pattern depended on, including @. Every email address in a feedback item was silently getting its @ removed and the rest left in the clear, which defeats the entire point of scrubbing it. It passed every test I'd written, because I'd tested the masking function in isolation, never in the actual order the pipeline calls it.

Accomplishments that I'm proud of

The agent actually closes the loop. It writes and opens a real GitHub issue on its own, on a schedule, with no one watching, rather than just flagging that one might be needed. Making that safe is the part I'm proudest of: duplicate-ticket prevention that holds up not only when two cycles race each other, but when Pub/Sub redelivers the same trigger forty minutes later.

The bilingual grouping also works the way I wanted. A French review and its English counterpart about the same bug end up in one cluster, so a founder reads one problem instead of two. And the two-model setup earns its keep: Gemma handles the cheap, high-volume triage, Gemini does the reasoning, and an automatic fallback keeps the whole pipeline running end to end even with the Gemma endpoint switched off.

What I learned

Language detection is a solved problem and doesn't need a language model. I originally had the 1B-parameter Gemma model guess a feedback item's language as part of its triage output, and it got it visibly wrong on short text (an English review classified as French). Pulling that one field out into a small deterministic library, run before anything touches a model, was both more accurate and removed an entire class of failure from the pipeline.

An idempotence guarantee is only as strong as the identity key behind it. I'd assumed "check if this thing already has a ticket" was enough; it wasn't, because what counted as "this thing" wasn't stable across two different senses: same content grouped differently by a non-deterministic label, and same content processed twice because a message got redelivered. Both needed a fix, and neither one would have caught the other.

The bugs that actually mattered, the PII leak and the duplicate tickets, weren't things I found by reading the code more carefully. I found them by running the real pipeline against real inputs and checking the actual output, not the output I expected. A test that only exercises one function in isolation can pass while the assembled system is quietly wrong.

What's next for feedbackai-agent

Authentication and real multi-tenancy. The data is already partitioned by project_id from write to read, including the vector search, so this is a next step rather than a rewrite. After that: action targets beyond GitHub, more ingestion sources behind a proper fetcher, and thresholds the agent tunes on its own.

Built With

Share this project:

Updates

Submission history