ProctorAI

ProctorAI: Live AI interviews you can trust, identity‑verified and 100% cheat‑proof.

Comment

ProctorAI — Secure AI‑Proctored Interviews

ProctorAI is an AI interviewer that makes hiring more accurate, smooth, reliable, and authentic—while enforcing anti‑cheating through continuous, real‑time monitoring. It verifies identity with face and voice baselines, runs a live OpenAI‑powered interview over WebRTC, and generates a deterministic, evidence‑backed evaluation with a proctoring impact score.

Inspiration

Remote interviews have become the norm, but integrity and consistency remain hard problems. Recruiters struggle to balance speed with rigor, while candidates worry about fairness. We were inspired to build a system that:

  • Protects authenticity: Ensure the person interviewing is the person hired.
  • Scales interviews: Run high‑quality interviews with live AI.
  • Reduces bias and noise: Use deterministic scoring and transparent proctoring signals.
  • Respects the candidate: Keep the experience smooth, responsive, and human‑feeling.

We wanted the best of both worlds: the empathy and structure of a great interviewer, and the integrity guarantees of a modern proctoring system.

What it does

End‑to‑end, in one flow:

  1. Context ingestion: Upload resume and job description; we extract relevant text for tailored questioning.
  2. Baseline verification: Capture face and voice baselines with lightweight, privacy‑conscious embeddings.
  3. Live interview:
    • Real‑time AI interviewer via OpenAI Realtime (WebRTC low‑latency speech).
  4. Continuous proctoring: Frame‑by‑frame and segment‑by‑segment analysis detects anomalies (unrecognized faces, multiple faces, unauthorized voice, etc.), with temporal smoothing to avoid false positives.
  5. Evaluation: Deterministic deduction scorecard + AI‑generated rubric. Final report includes strengths, growth areas, next steps, and a transparent proctoring summary.

Deliverables for each session:

  • Structured transcript, proctoring events, and a JSON evaluation report.
  • A single proctoring impact score that’s easy to interpret and audit.

How we built it

  • Stack: Django 5 + DRF, SQLite (dev), vanilla templates + JS orchestrator.
  • OpenAI Realtime (WebRTC): Backend mediates SDP offers/answers and ephemeral credentials, returning ICE servers and a client secret for the browser to establish a low‑latency voice session.
  • Evaluation: OpenAI Chat Completions with response_format = json_object to produce a consistent rubric, always overwritten with server‑side proctoring truth to prevent hallucinations.
  • Face verification: face_recognition embeddings with optional multi‑sample enrollment and a dynamic threshold.
  • Voice verification: Lightweight spectral embeddings + cosine similarity; WebRTC VAD if available, heuristic VAD alternative if not.
  • Temporal smoothing: Rolling windows for face and voice to suppress transient noise before logging anomalies.

- Deterministic scorecard: Converts proctoring signals into cause‑and‑severity deductions for a transparent final impact.

Challenges we ran into

  • WebRTC handshake + ICE: Getting SDP exchange and ephemeral secrets right, while handling browser permission flows consistently.
  • False positives in proctoring: Without smoothing, minor head movements or short audio artifacts could trigger anomalies; temporal windows fixed this.
  • Optional deps and builds: face_recognition/dlib and OpenCV can be heavy to set up across environments; we kept them lazy/optional.
  • SQLite lock bursts: Frequent event writes can lock the DB in dev; we added brief retry logic and lightweight rate limiting per session.
  • Strict JSON from LLMs: We enforced schema and added deterministic safeguards to guarantee UI stability.

Accomplishments that we're proud of

  • A polished, end‑to‑end demo: context → baselines → live interview → evaluation.
  • Cheat‑resistant by design: Identity‑bound voice and face, with transparent deductions.
  • Great UX: Modern dashboard, minimal friction, and clear statuses.
  • Deterministic scoring: Recruiters get a clear, auditable proctoring impact alongside an AI rubric.
  • Robust APIs and docs: Interactive Swagger and clean service boundaries.

What we learned

  • Small design choices (e.g., smoothing windows) can massively improve perceived fairness.
  • WebRTC with Realtime is powerful but demands careful handling of permissions, ICE servers, and network edges.
  • For evaluation, aligning prompts with actual transcript evidence is critical—never score what isn’t said.
  • Resilience and reliability build trust.

What's next for ProctorAI

  • Production data & infra: PostgreSQL, persistent baseline storage, and object storage for artifacts.
  • Multi‑tenant + SSO: Org workspaces, role‑based access, audit logs.
  • Advanced proctoring: Multi‑camera support, on‑device privacy‑preserving checks, better spoofing defenses.
  • Fairness & explainability: Bias audits, per‑signal calibration, richer rationales in the rubric.
  • Integrations: ATS/HRIS pipelines, webhooks, and exportable JSON/PDF reports.
  • Analytics: Team dashboards, cohort comparisons, and question bank insights.
  • Platform reach: Low‑bandwidth mode, mobile‑first UI, and accessibility improvements.

Appendix

  • API docs (Swagger): http://127.0.0.1:8000/swagger/
  • Core files of interest: core/services/{openai,interviewer,evaluation,context,scoring}.py, core/ai_utils.py, core/views.py, core/static/core/script.js.

Built With

Share this project:

Updates