Rubric Grader: Proposed Grades, Teacher Decides

Inspiration

Code generation is solved. What is still hard is everything that happens after the code exists: reviewing it, testing it, scanning it, shipping it and watching it run. I am a solo developer, so I wanted to know how much of that lifecycle I could hand to GitLab Duo agents without giving up control.

I picked a product with the same rule I wanted for the agents: a teacher grading short answers. A tool can propose a grade, with reasons, but only the teacher decides. The agents follow the same pattern: they propose a merge request, and I approve it.

What it does

Rubric Grader is a small tool for teachers with no classes, rosters or accounts.

1. A teacher writes an activity with a rubric (up to 10 criteria with points, optional keywords and minimum words) and gets two links: one for students, one private for the teacher.
2. A student opens the link and submits an answer. No account is needed.
3. The system proposes points for each criterion with a short justification. The rules are deterministic and explainable: no AI is involved in the grading itself.
4. The teacher reviews the answer, adjusts the points, adds a comment and publishes. An answer that was never reviewed cannot be published.
5. The student sees the result only after publication.

The live demo has a sample activity, so you can explore it without creating anything.

What is new: the repository was created on October 21, 2026, during the submission period, and every line of code was written for this hackathon, in Python, by the Duo agents from issues. The only thing carried over from my earlier project, SELITH EDU (a different stack: Next.js and Supabase), is the visual design system: the color palette, the system font stack and the card shapes. No code, no data and no files were copied.

How GitLab automates the post-code lifecycle

Every feature began as an issue. The GitLab Duo Developer agent read the issue, wrote the code and the tests, and opened a merge request. I reviewed each one. In total there are 13 merged merge requests, 14 closed issues, 58 pipelines and 173 automated tests.

All nine DevSecOps stages are touched:

• plan: features written as issues with acceptance criteria.
• create: the Duo Developer agent opens a merge request from each issue.
• verify: ruff and pytest on every merge request. When lint failed, I sent the exact error back and the agent fixed its own code instead of silencing the rule.
• package: a Docker image (non-root, port taken from the PORT variable) built and pushed to Artifact Registry.
• secure: SAST, secret detection and pip-audit. The audit found 8 known vulnerabilities in dependencies the agent had pinned from memory. Its sandbox had no network, so it said it could not verify new versions. We allowed the package index in .gitlab/duo/agent-config.yml and a second agent run fixed all eight.
• release: staging deploys automatically to Google Cloud Run with a smoke test. Production is a manual job that needs human approval.
• configure: CI/CD variables, Workload Identity Federation (no keys in the repository) and the Duo sandbox network policy, all as code.
• monitor: a scheduled pipeline checks the live service and, if it is down, opens an "Incident: health check failed" issue by itself, with the pipeline link.
• govern: protected main, human approval for production, and agents that state what they could not verify.

Autonomy level: Supervised. The agents carry out multi-step work; a person approves the outcome of every merge request and the move to production.

Challenges we ran into

Real problems, found and fixed by the same loop:

• An agent said a merge request was ready while the lint job was failing.
• Three agents each added their own include: block to .gitlab-ci.yml, so only the last one loaded and the security and deploy jobs silently disappeared. We merged them into one list.
• After a green deploy the page had no styles. Cloud Run hides HTTPS from the app, so the stylesheet URL was built with http and the browser blocked it. An agent fixed it with proxy headers and a relative path.
• The scheduled health check also ran the whole pipeline, including deploys. Limiting it to the check cut the runner time per scheduled run from about 4.9 minutes to 0.19.
• The first version stored data in a temporary file. We moved to Cloud SQL (PostgreSQL) with a SQLite fallback and a seeded demo activity.

Accomplishments that we are proud of

• A complete path from issue to production on Google Cloud, with no service-account keys in the repository.
• Security findings that the agents found, explained honestly and fixed.
• A monitor that opens its own incident issue.
• A tool teachers could actually use, with accessible pages and no JavaScript or external assets.

What we learned

Agents are good at executing a clear issue and bad at knowing what they could not check. The useful pattern was to make them say what they did not verify, check it ourselves, and send back the exact error. A short, precise comment fixed more than a long new request.

What is next

• Optional AI-assisted explanations on top of the deterministic grading, always behind teacher approval.
• Accounts and a teacher dashboard for people who want to keep many activities.
• Using the incident issue as a trigger for an agent that proposes a fix.

Built With

• artifact-registry
• cloud-sql
• docker
• fastapi
• gitlab-ci-cd
• gitlab-duo-agent-platform
• google-cloud-run
• jinja
• postgresql
• pytest
• python
• ruff
• sast
• secret-detection
• secret-manager
• sqlite
• workload-identity-federation