ÉquiAlgo

Inspiration

The brief opened with a number that sounded like success: a student-financing model that is 88% accurate. Then it showed the other number: the model grants awards to 48.4% of applicants from Montréal and Capitale-Nationale, and only 27.3% from Bas-Saint-Laurent, Côte-Nord and Gaspésie–Îles-de-la-Madeleine.

That gap is what drew us in. A high accuracy score sounds like proof of quality, but here it measured something else: how faithfully the model copied a human committee that was itself under audit. We wanted to find out how much of the gap was merit and how much was bias. Then we wanted to build something an institution could actually deploy and defend.

What we learned

Accuracy against biased labels rewards bias. The closer a model mimics the historical committee, the better it scores, and the more unfair it is.
Deleting a sensitive column does not delete the bias. Postal code maps one-to-one onto the region, and distance to campus predicts it almost perfectly. When we removed the region, the penalty simply moved into distance.
Sometimes you have to keep a variable in order to remove it. Training with an explicit remote-region indicator let one coefficient absorb the penalty, which we then set to zero when scoring.
Off-the-shelf fairness tools can be fooled by the labels. Measured against the committee's own decisions, both groups already looked equally treated, so fairlearn's equal-opportunity constraint had nothing to fix.
Choosing a fairness metric is a value choice. We chose equal opportunity, because a scholarship should give equally deserving applicants the same chance wherever they live. We also learned to say exactly why demographic parity holds in this data: once hours worked are counted, both groups have the same merit distribution. Nobody imposed a quota.
A bias can hide behind the one you were told to look for. The committee also favoured richer households by up to 1.9 R-score points, which the brief never mentioned.

How we built it

We worked in Python with pandas, scikit-learn, fairlearn and matplotlib, in a Jupyter notebook backed by a script.

Audit. We compared grant rates at equal R score, modelled the committee with a logistic regression, and measured the regional penalty in R-score points: 1.40, with a bootstrap confidence interval of 1.27 to 1.54. We ranked every column by how well it reveals the region.
Reconstructing merit. The reference standard is hidden, so we used the indicative scorer as an instrument. We ran 17 probe submissions, each changing one element of the rule. Merit turned out to be the R score plus 0.145 points per weekly hour worked. Income, distance, regional bonuses and quotas all made the agreement worse.
The fix. We removed the penalty, swept how much of it to remove to draw a Pareto front, and compared the result with fairlearn's mitigations. The final model reduces to one transparent rule: grant if R score + 0.145 × weekly hours ≥ 30.08. It funds 40.0% of both groups and agrees 94.6% with the reference, against 88.7% for the committee's own rule.
Governance. We researched Quebec's privacy law (Law 25), the Quebec Charter and the AMF's AI guideline. From that research we built:
- plain-language explanations for every applicant;
- a blind human-review band around the threshold;
- a label-free monitoring dashboard. The pipeline refuses to release decisions if any check turns red.

Challenges we faced

No ground truth. We could never see the reference standard, so every claim had to be checked against probe results or label-free metrics. Probe submissions were rate-limited, so each one had to test a single, well-chosen hypothesis.
Telling bias from real differences. Remote applicants have slightly lower R scores but work about four more hours a week. Deciding what counts as merit, and what counts as a value choice for the institution, took the most debate.
Law from outside Quebec. None of us is Quebecois. We had to learn which statutes apply to a fictional financial institution, cite primary sources, and be honest about what remains a question for a lawyer.
Monitoring without labels. In production nobody knows who truly deserved a grant. We designed checks that need no labels and calibrated their alert thresholds by simulation. We also tested them on failure scenarios. The dashboard catches biased human reviewers and tells population drift apart from bias.
Knowing when to stop. Near the threshold, the reference behaves almost like a coin flip. Once we confirmed that a more complex model fit no better, we spent our remaining time on fairness and governance instead of chasing leaderboard decimals.