FairScreen Demo for Qwen Cloud Hackathon

FairScreen is a chat and agent based hiring tool that anonymizes applicants, scores them, audits for bias, and sends fair decision recommendations to recruiters.

Comment

Inspiration

We started from a suspicion rather than a product idea: every résumé screening tool on the market claims it anonymizes candidates, and almost none of them can show you the receipt.

The moment that turned the suspicion into a project was small. We stripped a candidate's name and school from a profile the way a normal ATS does, then actually read the text that reached the model. It said "International public research university." The school name was gone and the candidate was still legible: studied abroad, high confidence, one phrase. Nobody wrote that leak on purpose. It survived because the check everyone runs is "did we remove the field", and the question that matters is "what can still be recovered from what the model saw".

So we built FairScreen around a different claim. Not "we removed bias", which no screening tool can honestly promise, but: here is the exact payload the scorer read, here is what an adversary model recovers from it, and here is the number.

What it does

FairScreen is a résumé screening pipeline with three parts, and the third is what makes the first two mean anything.

The Anonymizer redacts at three levels: standard (what real ATS tools do), abstracted (what we ship), and type_only (an experiment that drops institution tier entirely).

The Disclosure Buffer decides what the human reviewer may know and when. This is the part we care most about, and the reasoning behind it is simple: a reviewer who learns a name before deciding cannot un-learn it. Blindness is not a property of a database column, it is a property of a timeline.

The Auditor is an agent that attacks our own output: counterfactual twins, an adversarial attribute probe, and a rubric re-weighting.

Around it sits a Streamlit reviewer console where an HR user opens a blind packet, writes a reason, and makes a real decision.

How we built it, and why this way

We measured leakage instead of asserting it. A separate adversary model tries to recover protected attributes from the exact payload the scorer saw, never from ground truth, because a real candidate does not come with ground truth. Mean leakage: standard 0.423 with 9 of 20 candidates above 0.5, abstracted 0.173 with 4 of 20, type_only 0.185 with 4 of 20.

That third row is the honest part. Dropping institution tier fixes nothing. The residual leak is one word, International, and no rung of our ladder removes it. We shipped that as a finding instead of quietly tuning it away.

Every threshold is measured, never guessed. Re-scoring an identical, unchanged payload still moves the total by up to 0.0525, so the total gate sits at 0.1 and each criterion is gated on the worst wobble ever observed for that specific criterion. We found the noise floor is bimodal, which matters: a mean based threshold would have fired on 2 of 20 pure noise samples and we would have shipped roughly a tenth of our flags as noise.

We split two findings that look identical. Lowering a GPA should move education_credentials, because GPA is a credential and the rubric's author chose that weight. It must not move certifications_licensure, because none of that evidence changed. Movement outside an axis's legitimate channel is flagged contamination: rather than drift:, since a model letting one field recolour its reading of unrelated evidence is a different defect from a rubric doing its job.

Only the Auditor is an agent. This was a deliberate constraint and it cost us features. The scorer gets no tools and makes exactly one call per candidate, job and level. The auditor re-runs it hundreds of times to measure drift, and a scorer that reached for a tool on one pass and not the next would mix its own noise into every delta. A 0.12 drift has to be attributable to the swapped field or it means nothing. The agent picks which probes to run and writes the summary; every number in a report comes from probe output. Across 20 investigations its gate decision matched the agentless computation 20 out of 20. It chose the route, not the conclusion.

Checks run on the rendered artifact, not the code that built it. assert_blind() scans the dictionary handed to the console. Auditing the builder would only prove the builder is careful, which is not the claim worth making.

No framework. A loop, a status field on disk, and "the file exists so that step is done". LangGraph would have been a week of learning to replace forty lines of dispatch, and it would put a vendor's control flow between a reviewer and an audit trail meant to be read by a human.

Challenges we ran into

A blind console and a broken blindness checker report the same number. This is the scariest failure mode in the whole project, because it produces no artifact anyone would question. Our answer is --self-test on nearly every script: it plants a name and a school into a view and requires the check to fail. If the checker ever goes quiet, the build breaks.

The leak was in the ids, not the data. Our counterfactual variants are named things like c_fin_010__school-_womens_college. Printing candidate ids in the console would have spelled out the protected attribute on screen, in the one tool built to hide it. Candidates render as hashed handles like CAND-FD6345 instead.

We were rejecting people with a machine and did not notice. The orchestrator wrote rejected for anyone below the shortlist cut. Nothing was hidden, nothing was buggy, and the sentence "a human decides every rejection" read as true while being false in the exact place nobody thinks to check. Splitting the status into not_shortlisted (computed) and rejected (decided by a named person, with a reason, on a blind packet) was the whole fix. We deliberately ship no automatic cut, because validating one requires knowing who should have been hired, and this corpus has no such label. Precision and false negative rate here are not merely unmeasured, they are undefined.

Ordering turned out to be load bearing. The orchestrator treats hr_review as a wall and never re-reads a candidate's audit once they are parked there, so auditing has to complete before orchestration starts. Two scores are also only subtractable when produced under identical conditions, which means changing what the scorer reads invalidates all 348 scorecards and all 288 counterfactual deltas at once.

Accomplishments we are proud of

The two best moments in our demo are both the system refusing rather than succeeding. Type a decision reason containing a pronoun or a protected class and the write is refused in red, naming every hit, because HR was shown a blind packet and a reason describing something that was not on screen describes something the reviewer should not have. And in read only mode the forms stay live on purpose: you can fill one in, submit it, and watch the refusal come back from workbench.decide() rather than from a greyed out button. A disabled button is a claim about the page. We wanted a claim about the system.

What we learned

A guard that is also its own witness proves nothing. That single idea reshaped the codebase: stage_for() derives the stage from state on disk instead of trusting a caller, timeliness_violations() re-audits the disclosure log rather than trusting the function that writes it, and the blind packet is built by calling the scorer's own build_payload() so that "HR sees exactly what the model scored" holds by construction instead of by two field lists happening to agree.

We also learned to state gaps out loud. This buffers exposure, it does not eliminate bias: the moment an interview happens, a human sees a person. The corpus is synthetic, 20 candidates across 2 jobs, small enough that one candidate moves any rate. The console has no authentication. All of that is in our README rather than omitted from it.

What's next

The Communicator: an agent that sits between approval and disclosure. HR approves an interview while still blind, the agent reads the name and contact address, sends the invitation, and the name unlocks for HR only once delivery is confirmed. The point is conceptual, not technical: a machine handles the identifying data because a machine has no impression to form, and the human who cannot un-learn a name never needs it until there is a person to greet. The rung already exists in our disclosure ladder, waiting.

After that: replacing the US and International descriptor binary with reputation radius, which is the only reason four candidates still leak national origin at every rung, and which costs a full re-measurement of everything.

Built With

Share this project:

Updates