Repo Referee

Inspiration

In two long university team projects, one or two people translated the assignment into tasks and handed out the rest. Tasks went on a Trello board that nobody updated, because the real work happened in GitHub. The result: work split by who spoke up instead of who fits, nobody knowing how far anyone was, and at the end no clear answer to "did we actually meet every grading criterion?"

What it does

Repo Referee turns a project assignment and its grading criteria into small, fairly shared tasks in your team's GitHub repo, then shows by itself whether you are on track and whether the work is fair.

1. Create a project and add the team (name + GitHub username)
2. Paste the assignment. The AI drafts small tasks, each with a "done when", a weight from 1 to 5 and a linked grading criterion
3. Connect the GitHub repo. The app checks who can be assigned there
4. Review and choose: change anything, then everyone picks their own tasks. A bar per person shows their points against the fair share (total points ÷ team size), with a friendly heads-up when someone goes over
5. One click sends every task to GitHub as an issue, assigned to the person who chose it

After that nobody updates anything. The dashboard reads GitHub every 10 seconds: close an issue and its card turns green. It shows friendly feedback, progress per criterion, contribution per person weighted by task size, and a weekly summary written by AI from the closed issues. Issues closed as "not planned" drop out of the totals.

How I built it

A small local web app: a Flask (Python) backend, plain HTML, CSS and JavaScript, the Claude API for task generation and the weekly summary, and the GitHub REST API for issues. All Claude calls live in one module and all GitHub calls in another, and the calculations (fair share, progress, contribution, feedback) are pure functions covered by pytest. I built it with the Devpost Learn skill pack: scope, PRD, technical spec and a build checklist, all in the repo's devpost/ folder. The demo uses a fictional room-booking assignment.

Challenges I ran into

• Status must come only from GitHub. The app has no way to change a task's status, so "done" is always read from the issue.
• No duplicate issues. A hidden marker in each issue body lets a retry adopt issues that were already created, even when GitHub's answer got lost.
• People who haven't accepted the repo invitation can't be assigned. GitHub refuses with a 422, so the app warns first and can send anyway: those issues go out unassigned but still name the person.
• Keeping the AI useful. I tuned the prompt to make only the tasks an assignment really needs (usually 6 to 15). Generating the sample assignment costs about $0.03.
• A coach, not a referee. All feedback is positive and never names someone as behind.

Accomplishments that I'm proud of

• The whole loop works end to end with the real Claude API and real GitHub issues: paste an assignment, send the tasks, close an issue, and the dashboard follows within 10 seconds.
• Two independent review rounds found 28 problems in the code, and all of them were fixed.
• It feels like a finished product, not a technical demo: a step bar, a calm board per grading criterion, a start page and saved projects.

What I learned

Planning before building made the build much calmer. I also learned to test the risky parts (the AI call and GitHub's assignee behavior) first, and to let the AI propose while the team decides.

What's next for Repo Referee

Reading progress from the shared Word report, smarter task suggestions based on each person's skills and available time, and running the app online with real accounts. The "Share board" link already supports that once PUBLIC_URL is set.

Demo video: https://youtu.be/vHjSWE-AR2s Code: https://github.com/GitEmir/repo-referee

Built With

• claude-api
• css
• flask
• github-rest-api
• html
• javascript
• python