Local RLHF Optimization Engine

We built a local, multi-agent reinforcement learning pipeline using FastAPI and the Google Cloud Agent Builder (ADK). The engine utilizes Gemini 2.5 Flash for both a Policy Generation Agent and a Critic Agent.

Using the GRPO (Group Relative Policy Optimization) framework, the Policy Agent attempts to answer prompts under strict formatting constraints. The Critic Agent evaluates these completions, providing reasoning and scalar reward scores. The math engine then calculates the relative advantage of the outputs. To ensure full enterprise-grade observability, I used OpenTelemetry to stream the entire execution loop, token usage, and latency directly into a local Arize Phoenix dashboard.

Built With

• agent
• arize
• builder
• cloud
• fastapi
• google