-
-
Home
-
Live Work
-
Workflows
-
Agents
-
Incidents
-
Memory
Inspiration
We kept thinking about one problem with most AI products today: they are good at answering questions, but they usually stop there.
A real product team does not stop after finding an answer. It watches what users are doing, notices when something goes wrong, talks to users, investigates the cause, writes code, tests the fix, gets the right approval, ships it, and then asks a very important question:
Did it actually work?
That is what led us to build Product OS.
Our inspiration also came from Andrew Ng's way of thinking about AI systems as loops instead of one big model doing everything. We took that idea and applied it to product development as three connected loops:
- Product loop: observe users, analytics and reliability signals, then find problems or opportunities.
- Engineering loop: investigate, write code, test it, review it and ship it.
- Learning loop: measure what happened after the change and feed that knowledge back into the next cycle.
The important part is that these loops connect.
A customer complaint can become an investigation. The investigation can become a code change. The code change can become an experiment. The experiment becomes new product knowledge.
So instead of building another chatbot, we wanted to build something closer to an autonomous product team.
The core idea became:
Observe the product → understand users → identify problems or opportunities → coordinate the right agents → ship changes → measure impact → learn → repeat.
Product OS is built around this loop. It detects a signal, investigates it, brings in customer evidence, identifies whether it is a bug or a product opportunity, coordinates the right work, and remembers the outcome.
What it does
Product OS is an AI product operations team.
It watches product signals such as analytics, logs, customer feedback and product events. When something unusual happens, the Signal Agent triggers an investigation.
Different agents then handle different parts of the problem:
- Signal Agent detects unusual behavior
- Investigator Agent gathers and correlates evidence
- Analytics, Logs and Deployment Agents investigate different sources
- Customer Voice Agent talks to affected users
- Feedback Agent turns conversations into structured evidence
- Root Cause Agent forms a likely explanation
- Code Agent investigates the repository and proposes a fix
- Test Agent creates and runs regression tests
- Product Agent evaluates feature opportunities
- Risk Agent decides what can be automated
- Learning Agent measures the result and stores what was learned
The agents have different responsibilities and coordinate with each other. ADK 2 workflows let us compose deterministic nodes and AI agents into a graph, including parallel investigation paths.
For example, imagine checkout conversion suddenly drops.
Product OS sees the analytics signal, correlates it with logs and a recent deployment, investigates whether a particular device or browser is affected, and then asks the affected customer what actually happened.
Instead of a generic survey question like:
"Why didn't you complete payment?"
the Customer Voice Agent has context such as the device, transaction amount, failure type, previous attempts and known issues, then asks follow-up questions that gather useful diagnostic evidence.
The conversation is converted into structured evidence instead of just being saved as a transcript.
The system then combines:
customer feedback + analytics + logs + deployment timing + browser segmentation
to form a root cause hypothesis.
From there, the Code Agent investigates the repository, creates an implementation plan, generates a regression test, runs tests and produces a PR.
The Product side works the same way.
If many customers ask for the same capability, the Product Agent clusters the feedback, estimates impact, creates a product proposal and sends it into the normal product and engineering process.
The system also has a verification loop.
After a change ships, Product OS monitors the result, compares metrics and customer feedback, and asks:
Did the problem actually disappear?
That closes the loop instead of ending at "PR merged."
How we built it
The core agent layer uses Google ADK 2, with workflows for coordinating deterministic steps and AI agents. The architecture also leaves room for A2A so specialized agents do not all need to live inside one giant process.
The system is organized into a few layers.
Signal layer
Product signals come from analytics, ads, Firebase, logs, support, voice and application events.
We treat analytical data differently from memory.
BigQuery is for facts such as:
"Payment conversion was 81.7%."
Memory is for knowledge such as:
"Release 4.3 caused a payment regression and was fixed by PR #184."
That distinction matters because we do not want the memory system to become a dumping ground for raw analytics.
Agent layer
The main workflow is:
Signal → Investigation → Evidence → Root Cause → Bug or Feature → Execution → Review → Learning
For bugs, the engineering path goes through code investigation, implementation, testing and PR creation.
For product opportunities, the path goes through hypothesis, impact analysis, experiment design and engineering work.
Tool layer
The agents connect to external systems through APIs and MCP.
The architecture is designed around tools such as GitHub, Gmail, Calendar, Drive, Docs, BigQuery, Firebase, analytics systems and customer support systems.
This is also where the product becomes useful as an actual product team instead of just an agent demo.
For example, a developer review request becomes:
PR needs approval → identify owner → check calendar → find availability → schedule review → send notification.
Security layer
We did not want to solve agent safety by writing "please do not do dangerous things" in a prompt.
The architecture uses identity, policy and gateway controls around actions.
The idea is:
Agent → Identity → Permission check → Gateway → Model Armor / data protection → Tool
So the answer to:
"What stops the engineering agent from accessing customer data?"
is not:
"The prompt tells it not to."
It is:
"It does not have the identity and permission to access it."
The Risk Agent also separates low, medium and high risk actions. Documentation fixes and low-risk changes are automated, while business logic, payment flows and database changes require developer approval. High-risk actions such as authentication, payment authorization, financial calculations and destructive database changes require human approval.
Model Armor and sensitive data protection are especially important because the system touches customer data, payment-related events, voice transcripts, email, GitHub and production systems.
Memory
We also wanted Product OS to remember lessons, not just retrieve documents.
The memory model is split into:
- Customer memory
- Product memory
- Engineering memory
- Organizational memory
For example, after an incident the system remembers the root cause, affected environment, fix, PR and what should be tested next time.
Later, a similar problem brings that knowledge back into a new investigation.
Product interface
We wanted the interface to make the agents feel like a real team at work.
Instead of only showing chat bubbles, Product OS has rooms, agent views, approvals, memory and a visual campus with working agents.
The idea was inspired by products that make collaboration visible, but we wanted to apply that idea to autonomous agents.
The final interface deliberately became lighter and more welcoming instead of looking like a dark "war room". We also built the office inside the product rather than relying on an external document.
Under the hood, we used a Next.js console with a FastAPI backend and deployed the hosted system on Cloud Run.
We also had to make the architecture work in the real world, including static export routing, same-origin WebSockets, Cloud Run packaging, CORS, OAuth and the difference between development and production environments.
Challenges we ran into
One of the biggest issues was the boundary between Next.js static export and a live backend. The hosted console needed same-origin API and WebSocket behavior, while local development worked differently. We also had issues where a local API URL could accidentally get baked into the production JavaScript bundle.
Deployment had its own problems. The usual Cloud Run source deployment path was not available in this environment, so we had to build a packaging and deployment path using a hosted image and a generated tarball. Python package compatibility also mattered because the local environment and Cloud Run used different Python versions.
The UI had real problems too.
The campus pins looked correct on desktop but were completely wrong on some phone layouts because the image was being contained inside a different aspect ratio. We had to measure the actual image content box rather than positioning everything relative to the whole frame.
The pixel agents had similar problems. CSS overflow and animations could make agents look like their heads were being cut off. Some mobile room layouts also broke when too many agents were placed in one row. These were small bugs, but they taught us that an agent interface still has to survive normal frontend engineering problems.
Another challenge was learning ADK 2 properly instead of just carrying old patterns forward.
ADK 1.x patterns such as ParallelAgent and LoopAgent are not a drop-in match for the newer Workflow model, so we had to rethink the orchestration around Workflow, JoinNode, RequestInput and typed inputs.
That process actually became useful beyond the project itself, because it led us to contribute fixes and documentation back upstream.
Accomplishments that we're proud of
The biggest accomplishment is that Product OS became more than a concept.
It is a working product with an agent architecture, visual interface, workflows, memory, approvals, integrations and a deployed system.
We are especially proud that the system is designed around the entire product loop, not only the AI part.
It can connect:
signal → investigation → customer evidence → root cause → code or product decision → review → release → measurement → memory
That is the part we think makes Product OS different from a standard agent demo.
We are also proud of the safety model. The system is designed so that autonomy is not binary. Different actions have different levels of risk, and higher-risk actions stop at a human approval gate.
We also contributed back to Google ADK
While building Product OS, we did not just study Google ADK.
We filed real issues and pull requests based on problems we found while working with the technology.
Here are the contributions:
1. Model Armor integration docs
We added an integration page covering installation, configuration and limitations for Model Armor, and connected it to the safety documentation.
2. CLI deprecation documentation
We added a deprecation notice for older SequentialAgent, ParallelAgent and LoopAgent patterns and pointed developers toward the newer Workflow samples.
3. Safety plugin migration pointer
We updated the safety plugin README to point users toward the first-party ModelArmorPlugin while keeping the sample useful as a custom hook reference.
4. Model Armor tool-output sample
We added a Model Armor tool-output sample using ToolOutputModelArmorPlugin with an after_tool_callback, along with unit tests.
5. First-party tool-output screening
This was a feature request for a first-party Model Armor extension or companion for tool-output screening in ADK.
6. ADK 2 migration guide
We created a migration guide showing how older workflow patterns map to Workflow, JoinNode, loops and Workflow-as-Tool in ADK 2.
7. node_as_tool sample documentation
We added the node_as_tool workflow sample to the related samples section so developers can actually find it while reading the Workflow docs.
So this project was not only about building something on top of the ecosystem. We also used what we learned while building it to contribute improvements back to the ecosystem.
The contribution log records seven upstream items across the three repositories.
What we learned
The biggest thing we learned is that building agentic systems is much more about orchestration, state, permissions and feedback loops than it is about writing clever prompts.
A good agent does not solve the whole problem.
A good system decides:
which agent should act, what evidence it needs, what tools it is allowed to use, what should happen next, when a human needs to be involved, and what the system should remember afterward.
We also learned that structured evidence is much more valuable than raw conversations.
A customer voice interaction should not end as a transcript. It should produce useful information that another agent can reason about and combine with analytics, logs and deployment data.
We learned that memory needs structure too.
Customer knowledge, product decisions, engineering incidents and organizational practices are different kinds of information and should not all be treated the same way.
We also learned that autonomy needs boundaries.
Giving an agent access to more tools is easy. Giving it the correct permissions, safety checks, approval rules and auditability is much harder.
And finally, we learned that the real metric for a system like this should not be:
"How smart is the agent?"
A better question is:
How quickly can we move from a real-world signal to a validated improvement?
That is why we like the idea of idea-to-impact time.
The goal is not to automate humans out of the process. The goal is to reduce the time between discovering something important and learning what to do about it.
Product OS in Practice
The learning loop is a core part of Product OS.
Product OS processes product signals from analytics, customer feedback, reliability data and real application events.
The experiment side is also a first-class part of the system.
Instead of saying:
"We think this feature will help."
Product OS turns that into a hypothesis, designs an experiment, coordinates implementation, releases to a controlled percentage of users, measures the result and makes a decision based on evidence.
That creates the loop we originally wanted:
Detect → hypothesize → build → experiment → measure → learn.
The agent fleet is modular through A2A, so specialized agents can be discovered and coordinated instead of everything being implemented inside one application.
Product OS is an autonomous product operating system that sits between the product, its users and the engineering organization.
It continuously answers three questions:
What is happening?
What should we do about it?
Did what we did actually work?
That is the product we built.
Built With
- adk
- antigravity
- bigquery
- cloudrun
- cloudstorage
- fastapi
- firestore
- gemini
- genai
- github
- githubactions
- google-cloud
- modelarmor
- nextjs
- oauth
- pubsub
- python
- react
- sqlite
- tailwind
- terraform
- twilio
- typescript
- vertexai
- websockets
Log in or sign up for Devpost to join the conversation.