Rungs

Inspiration

Large enterprises already write their HR standards down. There is a job architecture, a grade rubric that defines what each level means, and a course catalog with hundreds of entries. All three exist as spreadsheets, and nothing connects them.

So the connecting is done by hand. A learning-and-development planner opens the three files, forms a judgement about where a person stands, and picks courses that feel about right. For a few hundred employees that is weeks of work, the reasoning is never written down, and when a manager asks why one person got four courses and another got two, there is no answer that survives the question.

What it does

First, it reads the paper. An employee record does not start as a row in a system. The annual self-assessment is a printed form, filled in by hand and returned to HR as a scan or a phone photo, and for a new joiner there is no row at all until somebody types one. Drop that scan on the dashboard and gemini-3.5-flash transcribes it under a schema; the printed job title is then matched against the job architecture and the course names against the catalog in code, because a model that guesses a course id produces a plan citing a course that does not exist. What the page did not say stays unsaid — on our sample form the years-of-service box is blank while the free text mentions "two years ago", and the record reports not stated rather than the 2 a helpful reader would write in. Below that point the pipeline cannot tell a person read off paper from a person read out of the roster file.

Then, for that person or any of the forty already on the roster, it runs four steps without further input.

Diagnose. It reads the person against the grade rubric and estimates their current level on each of three axes — Expertise, Problem-Solving, Innovation & Learning — citing only facts that were actually in the input, and stating what information was missing.

Compute the gap. Target grade minus estimated grade, per axis, with the target competency statements attached.

Plan. It assembles a twelve-week roadmap from the real course catalog, addressing the largest gap first.

Verify. It checks the plan against the training budget, the learning-load ceiling, the difficulty bounds for that person's grade, how many courses run concurrently, and whether every gapped axis is actually covered. When something fails, the plan goes back to the planner with the violations attached and is rewritten. Up to three passes.

Then it does the same for the entire roster in parallel and aggregates the result into an organisation-level view: which job families sit furthest from their targets, and on which axis. In our run, forty employees produced a ranking where the HR function itself carried the largest per-head gap.

How we built it

The pipeline is a Google ADK SequentialAgent containing an LlmAgent, a deterministic BaseAgent, and a LoopAgent that wraps a planner and a critic.

The design decision that matters: the arithmetic is not an LLM call. Gap computation and plan validation are plain Python inside BaseAgent subclasses. A system that decides who gets which training has to survive being questioned, and that means the same input has to produce the same number every time. So the gap math, the budget totals, the learning-load totals, the week-overlap check, and the coverage check are computed. The model does the two things only a model can do — judging where a person stands from their record, and sequencing courses into something coherent.

Every LLM output is constrained by a pydantic output_schema, so no stage parses free text.

A second reader, on a different model family. Organisations that grade people for a living do not let one assessor decide alone. diagnose (Gemini 3.5 Flash, on Vertex) and a second reader (Gemma 4 31B, through the Gemini API) sit in an ADK ParallelAgent over the same record and the same rubric, and neither sees the other's answer — running them in sequence would let the second anchor on the first, which is the exact failure a second reading exists to prevent. A deterministic reconcile step compares them and puts the contested ones in a review queue. An L&D team cannot re-check four hundred assessments; they can re-check the ones where two readers disagreed.

Gemma has no structured-output mode, so unlike every Gemini stage its JSON is parsed defensively, and the whole step fails open: without a key, or on an error, the run completes as a single-reader assessment and says so. A second opinion that can take down the primary assessment is worse than no second opinion.

• Model: gemini-3.5-flash on Vertex AI, location: global — text and the scanned form
• Second reader: gemma-4-31b-it via the Gemini API
• Framework: Google ADK 2.8
• Infrastructure: Cloud Run (asia-northeast3), Firestore (catalog, memory bank, audit log, scanned intakes), Cloud Logging (structured JSON events per pipeline step)

A second reader, and what it told us about the first

Organisations that grade people do not let one assessor decide alone. Two readers go over the same file, and what needs a human is where they part company.

So diagnose (Gemini 3.5 Flash) and a second reader (Gemma 4 31B) sit inside an ADK ParallelAgent over the same record and the same rubric, each blind to the other's answer. Running them in sequence would let the second anchor on the first, which is the exact failure a second reading exists to prevent.

Scoring both against the planted set produced a result we did not expect:

Reader	Recall	False alarms	Precision
Gemini 3.5 Flash	8/13 (62%)	4/86 (5%)	67%
Gemma 4 31B	13/13 (100%)	2/86 (2%)	87%
Both readers agree	8/13 (62%)	0/86 (0%)	100%

The smaller open model read this task better. Every one of Gemini's five misses is the same shape — the evidence supported a higher grade than the record assigned, and it deferred to the record. Gemma took the evidence at face value in all five. Gemini stays the primary reader, since it produces the schema-constrained output the pipeline consumes and plans the roadmaps, but we are not going to report our own numbers selectively.

The routing rule took two attempts. The first was the obvious one: the readers disagree, so a person should look. We shipped it, scored it, and found flagged runs were 0.6× as likely to carry a real mismatch as unflagged ones — no signal, slightly worse than chance. The error was comparing the readers to each other rather than to the thing under question. Sorting each axis by how many readers depart from the grade on file separates them cleanly: both departing is a finding at 100% precision, exactly one departing is the contested case worth a person's time, neither departing means the record stands.

That is the review queue. An L&D team cannot re-check four hundred assessments. They can re-check the contested ones, and now they know which those are.

Challenges we ran into

gemini-3.5-flash is not served from regional Vertex endpoints. Requesting it against us-central1 returns a 404 with a message about the publisher model not being found, which reads like a permissions problem and is not. It has to be called with location: global.

Cloud Run's frontend reserves /healthz and answers it with its own 404 before the request reaches the container, so the health endpoint had to move to /api/health.

Cloud Run also throttles CPU after a response is returned, which silently kills background work. The batch endpoint completes forty employees only because the service runs with CPU always allocated — and even then, replacing an instance mid-batch takes the in-flight work and the in-memory job record with it, which a deploy landing during a roster run demonstrated. There is now a synchronous endpoint for anything driving the roster from outside, so the caller learns each outcome and can retry the one that failed.

Adding the Gemma reader broke every run until we worked out why: google-genai reads GEMINI_API_KEY and GOOGLE_API_KEY straight out of the environment. Mounting the Gemma key under either name does not just configure the second reader — it puts an API key in front of the primary Vertex path as well. The key now arrives under a name the SDK ignores. Finding that took far longer than it should have, because ADK runs the parallel read pass in a TaskGroup and the only thing surfacing was "unhandled errors in a TaskGroup (1 sub-exception)". Failures are now flattened through their sub-exceptions before logging.

And gemma-3-27b-it, which the docs led us to, is not served on this endpoint at all. The models actually available are gemma-4-31b-it and gemma-4-26b-a4b-it. The enablement script checks the model answers before wiring anything to it.

Accomplishments that we're proud of

We measured the things that are easy to claim and hard to earn, changed the design when the measurement disagreed with us, and then found out that we had been measuring wrong.

A single run is not a measurement. Every number we had was taken from one pass over the roster. Scoring the same forty records a second time returned 62% recall where the first had returned 77%. Both were true; neither was a result. So the measurement is now ten full passes — 400 assessments against one Cloud Run revision — reported with the spread, and the harness that produces it is in the repository. It needs no Google Cloud credentials: it drives the public URL and scores against an answer key that ships with the code, so anyone reading these numbers can recompute them.

Thirteen of the forty people carry a planted mismatch: their self-assessment describes work one level above or below their assigned grade, on one axis. The roster the agent reads does not say which, and the answer key sits in a file nothing in the pipeline loads.

Recall 72.3% (sd 9.0, range 62–92) at 86.7% precision, with false alarms on 1.8% of the eighty-six clean axes. That nine-point spread is the honest headline, and it is why the single-run numbers we started with were worth so little.

The misses are systematic, not noise. Four axes account for 34 of the 36 misses across ten passes, and one of them was missed in all ten. Every one is the same shape: the evidence supported a higher grade than the record assigned, and the reader deferred to the record. The system pulls toward the grade on file, and it fails by under-calling promotion cases rather than over-calling them. The safer failure. Still a failure, and now a named one.

The second reader turned out to be the better reader, and not by a clean win. Gemma 4 31B, scored against the same key over the same ten passes, finds 98.4% of the planted mismatches to Gemini's 72.3% — but at 82.0% precision against Gemini's 86.7%, calling roughly twice as many clean axes wrong. One conservative reader and one sensitive one. Two readers that fail differently is what makes the second call worth making.

The routing rule was wrong the first time, and measuring is how we found out. The obvious rule — the readers disagree with each other, so a person should look — scored at 0.6× chance. No signal. The mistake was comparing the readers to each other instead of to the thing in question. Routing on how many readers depart from the grade on file instead: axes both readers challenge carry a planted mismatch 99.0% of the time, and across ten passes and roughly 1,200 axis judgements, not one planted mismatch ever survived both readers unchallenged. Where the two agree the record stands, the record was right every time. That is what makes a review queue of twelve usable instead of decorative.

Confidence tracks evidence rather than decorating the output, and it is the steadiest thing we measured: 0.761 (sd 0.005) where a self-assessment exists against 0.242 (sd 0.015) where the record is blank. Whatever else moves between runs, the assessment's willingness to admit it does not know does not move.

The critic sends back 38% of plans — 190 of the 500 the deployed service has produced, read back out of the audit trails rather than asserted. All 500 satisfied every constraint in the end; 140 drafts broke the learning-load ceiling and 117 the budget. The planner is told those constraints in its own instructions, immediately above the data. That is the argument for making the verifier a separate deterministic step instead of another sentence in the prompt: stating a constraint is not enforcing it.

The assessment refuses to guess at gender. The model initially read the roster's names and wrote "she has completed…" — the records state no such thing. In a document that feeds a promotion decision, inferring a protected attribute from a name is a liability. It now refers to people by name or as "they", and a test checks for it.

The diagnosis states its own limits. Each estimate carries a confidence value and the agent returns a list of what it would have needed to judge better. An HR system that hides its uncertainty is worse than useless.

What we learned

Putting a deterministic verifier behind a generative planner changes what the system is. Without it, the output is a suggestion. With it, the output is a plan that satisfies stated constraints, and the constraint violations are the interface between the two.

What's next for Rungs

Reading the estimate against the person's own self-assessment rather than inferring from tenure and course history. Feeding completion outcomes back so the memory bank informs the next diagnosis. Letting an L&D planner adjust a constraint and see the whole roster replan.

Data statement

Hanwoori Group is a fictional conglomerate created for this project. Every job, competency statement, course, and employee is generated by a seeded script in the repository. The scanned self-assessment used to demonstrate intake is drawn by scripts/gen_form.py and put through what a scanner does to paper — nobody's real form appears anywhere. No real organisation, employee, or dataset appears in the code, the deployed application, or the video.

Built With

• cloud-logging
• cloud-run
• cloud-storage
• fastapi
• firestore
• gemini-3.5-flash
• gemma-4
• google-adk
• google-genai
• lyria-3
• pydantic
• python
• secret-manager
• veo-3.1
• vertex-ai