Autopilot Recruit

Inspiration

Traditional candidate screening is plagued by two main issues: recruiter burnout and unconscious bias. Recruiters spend hours scanning hundreds of resumes, leading to screening fatigue where qualified candidates from unconventional backgrounds are often overlooked. At the same time, unconscious biases regarding gender, graduation years, university prestige, or career gaps frequently skew candidate evaluations.

We asked ourselves: Could we build an autonomous recruitment agent that acts as a double-blind, fair hiring partner?

Our vision was to build an agentic hiring workflow that:

Strips all demographic and personal identifiers from the start. Evaluates candidates purely on merit (skills, experience, education match). Employs an independent "Bias Auditor" agent to check the scorer's math and reasoning. Keeps humans in the loop for borderline cases while letting high-confidence candidates auto-advance straight to a calendar booking.

What it does

Autopilot Recruit: Autonomous, Fair, and Multi-Agent Hiring Funnel Autopilot Recruit is a recruitment automation pipeline built for the Qwen Hackathon (Track 4: Autopilot Agent). The project integrates a series of Qwen-powered agents with a deterministic scoring router to intake resumes, redact personally identifiable information (PII), audit candidate scoring for biases, generate customized interview questions, draft outreach emails, and coordinate bookings on a shared interview calendar.

🤖 The Agents Autopilot Recruit is powered by 8 specialized agents collaborating to guide candidates from intake to calendar booking:

Job Extractor Agent (

job_extractor.py ) Role: Parses a raw, unstructured text description of a job opening. Task: Extracts structured properties like primary role name, minimum required years of experience, core technical skill lists, and education parameters into a clean database entity. Anonymizer Agent (

anonymizer.py ) Role: Enforces data privacy and double-blind meritocracy. Task: Redacts all personally identifiable information (PII) including candidate names, contact details, dates, company/school names, and gender pronouns, replacing them with standardized placeholders (e.g. [UNIVERSITY_A]). Extractor Agent (

extractor.py ) Role: Structures unstructured resume text. Task: Extracts technical skills, education history, total years of experience (as a decimal float), timeline red flags, and assigns an extraction confidence score. (Invoked twice: first on raw resume to track metadata, then on anonymized resume to extract unbiased qualifications). Scorer Agent (

scorer.py ) Role: Matches the anonymous candidate qualifications against job requirements. Task: Grades the candidate on a scale of 0-100 and computes an assessment confidence score ($C_{scr}$) based on profile detail completeness and red-flag occurrences. Bias & Fairness Auditor Agent (

auditor.py ) Role: Guardrail against unconscious/systemic bias. Task: Cross-checks the Scorer's rationale against the candidate's raw profile. Adjusts the score ($S_c$) upwards if the scorer penalized the candidate for life/employment gaps, non-FAANG prestige gaps, graduation dates, or gender-related hints. Question Designer/Generator Agent (

question_generator.py ) Role: Prepares the human interviewer. Task: Crafts personalized warm-up questions, deep-dive technical questions targeting candidate skills, and polite behavioral questions checking resume red flags/employment gaps. Scheduler Agent (

scheduler.py ) Role: Coordinates time-booking. Task: Scans available interview slots in the mock calendar, selects the 2 or 3 best-fitting dates, and drafts a friendly outreach email containing the calendar proposals. Q&A Agent (

qa.py ) Role: Candidate-facing customer support portal. Task: Dynamically answers candidate questions regarding work environment, benefits, PTO policies, remote setups, and pipeline interview stages based on official company policy context.

How we built it

We built the system as a decoupled, multi-agent architecture using FastAPI for the backend, a React (Vite) + TS frontend, and SQLite to log decision traces and candidates. All agent calls run through the Qwen Cloud API (qwen-plus) using OpenAI-compatible JSON schemas.

1. The Screening Workflow Pipeline [Resume Input] ──> Extractor Agent │ ▼ Anonymizer Agent ──> Extractor Agent │ ▼ Scorer Agent │ ▼ Auditor Agent │ ▼ [Tailored Questions] <── [Scheduler Agent] <── Router ├─> Auto-Reject (Low) ├─> Auto-Advance (High) └─> Human Review (Borderline) Intake & Extraction: Raw resume text is parsed. The Extractor Agent extracts the candidate's real name. Double-Blind Anonymization: The Anonymizer Agent replaces PII with bracketed placeholders. Anonymized Re-Extraction: The resume is re-extracted to capture structured skills, education, and red flags, ensuring no identifying metadata is preserved. Scorer Agent: Evaluates qualifications against the job description (skills, years of experience, education). Auditor Agent: Evaluates the Scorer's reasoning for biases, correcting the score if a candidate was unfairly penalized. Deterministic Router: Determines the candidate's path based on the audited score. Question Designer: Drafts interview questions tailored to candidate-specific red flags and skills. Scheduler Agent: Suggests slots from the mock calendar and writes a custom booking email. QA Assistant: Serves a public portal answering candidate inquiries about benefits or requirements.
2. Mathematical Formulation & Routing Logic We modeled the screening routing mathematically to guarantee deterministic, auditable decisions. Let:

$S_{raw} \in [0, 100]$ be the suitability score assigned by the Scorer Agent. $B$ represent the bias assessment vector (capturing age, prestige, gap-year, or demographic flags). $S_c \in [0, 100]$ be the audited, adjusted score for candidate $c$: $$S_c = \text{AuditorAdjust}(S_{raw}, B)$$ $C_{ext} \in [0, 100]$ and $C_{scr} \in [0, 100]$ be the extraction and scoring confidence levels respectively. $T_{rej} = 40$ (auto-reject threshold), $T_{adv} = 80$ (auto-advance threshold), and $T_{conf} = 60$ (minimum confidence threshold). The final routing status $f(c)$ for a candidate $c$ is determined deterministically by: $$ f(c) = \begin{cases} \text{needs_review} & \text{if } C_{ext} < T_{conf} \lor C_{scr} < T_{conf} \ \text{auto_rejected} & \text{if } S_c < T_{rej} \text{ and } C_{ext} \ge T_{conf} \land C_{scr} \ge T_{conf} \ \text{auto_advanced} & \text{if } S_c \ge T_{adv} \text{ and } C_{ext} \ge T_{conf} \land C_{scr} \ge T_{conf} \ \text{needs_review} & \text{otherwise} \end{cases} $$

Challenges we ran into

🚧 Challenges We Faced PII Leakage in Sub-prompts: During early development, the Scorer agent could sometimes guess demographics through names or specific dates. We solved this by implementing a strict double-extraction architecture: extracting the name first, passing the resume to the Anonymizer, and then passing the anonymized text to subsequent evaluation stages. Strict JSON Schema Enforcement: LLM outputs occasionally had malformed JSON brackets or missing fields. We addressed this by running a structured Pydantic parser around the raw completions. We implemented a 3-retry validation loop with feedback prompts on schema exceptions, achieving a 100% JSON parsing success rate in production. Ambiguity and Conflicting Data: Reconstructed dates in CVs are notoriously messy. When candidate timelines conflicted (e.g., overlapping full-time jobs at distant locations), the Scorer's confidence dropped. We mapped this directly to our deterministic router so that any evaluation with confidence $< T_{conf}$ is automatically escalated to human review, preventing silent AI hallucinations.

Accomplishments that we're proud of

Robust Multi-Agent Orchestration: Successfully tied 8 specialized agents into a fast, linear, and error-resilient recruiting pipeline. An Effective Double-Blind Guardrail: The combination of the Anonymizer Agent and the Bias & Fairness Auditor Agent successfully shields evaluations from subjective biases (e.g., restoring score equity for candidates with maternal/paternal gaps or state school degrees). Bulletproof JSON Recovery: Created a schema validation loop that recovers automatically from LLM parsing errors, ensuring the database stays clean and pipeline steps do not crash. Complete Audit Transparency: Designed a comprehensive decision trace logger recording system messages, prompt parameters, raw completion details, and validation error messages in SQLite, creating a highly auditable workflow dashboard.

What we learned

Separation of Reasoning and Routing: Relying on LLMs for routing leads to unpredictable behaviors. By separation of roles—using LLMs for analysis ($S_{raw}$ and $C_{scr}$) and hard-coded Python logic for routing—we created a pipeline that is both flexible and predictable. Double-Blind Auditing: Designing the Bias & Fairness Auditor agent opened our eyes to how AI can actively enforce compliance. The Auditor successfully catches and corrects penalizations from career gaps (e.g. maternal/paternal leave or layoffs), restoring candidates to their rightful scores. Audit Trails for Trust: Having a complete database record of every agent's prompts, completions, and token counts builds massive user confidence. Transparency is the only way recruiters will ever trust AI copilots to make autonomous decisions.

What's next for Autopilot Recruit

Production Calendar Integrations: Connect our scheduling mock calendar to live external APIs like Google Calendar and Microsoft Graph to verify real interviewer availability. Outbound Mail Delivery: Transition draft notifications to real candidate correspondence via SendGrid or standard SMTP services. Bias & Fairness Compliance Analytics: Add an analytics module that measures correlation coefficients between protected attributes (extracted prior to anonymization) and final scores, outputting bias telemetry charts. Multi-Role Generalization: Extend job extraction so the recruitment team can manage multiple job listings simultaneously, routing candidates dynamically to corresponding scoring profiles.

Built With

• fastapi
• python
• qwen
• react
• sqlite
• typescript