PrepPilot

PrepPilot: Product Journey & Evidence-First Multi-Agent Architecture

Inspiration

Interviews are specific. Interview preparation usually is not.

A candidate is not preparing for an abstract job. They are preparing for one specific role, with one specific background, on a specific date. Yet much of interview preparation still begins with generic question lists, generic frameworks, or polished AI-generated answers.

That led to the question behind PrepPilot: What if interview preparation started by understanding the gap between what this job requires and what this candidate can actually prove?

The problem becomes even more important with generative AI. It is easy for AI to produce an impressive interview answer. It is much harder to make sure that answer remains grounded in what the candidate has actually done. In a job interview, sounding better is not useful if the candidate cannot defend the story when someone asks the next question.

We wanted to build something different. PrepPilot would not try to manufacture the perfect candidate. It would help a real candidate understand their strengths, confront their gaps honestly, practice the difficult questions, and communicate their actual experience more effectively.

That became the idea behind PrepPilot:

Rehearse the interview before it happens.

What it does

PrepPilot turns three pieces of context, a resume, the actual job posting, and the interview date, into a personalized interview preparation journey.

It starts with a free Readiness Brief. PrepPilot breaks the posting into individual requirements and searches the candidate's resume for evidence of each one. Instead of returning a mysterious compatibility score, it shows which requirements are strongly evidenced, partially evidenced, or not evidenced. Where evidence exists, PrepPilot can point back to the candidate's actual experience. Where it does not, PrepPilot says so.

The brief then turns that analysis into action. It identifies the candidate's strongest positioning, surfaces genuine gaps, suggests honest ways to discuss those gaps, identifies questions worth preparing for, and creates a preparation plan based on the time remaining before the interview.

The Interview Studio goes deeper. It prepares twelve questions tailored to that specific candidate and role, including role-specific questions, questions about gaps, behavioral questions, and questions the candidate can ask the employer. Instead of simply displaying questions, PrepPilot explains why each one matters, what it may test, how the candidate can approach it using their own experience, and what traps to avoid.

Then the candidate actually practices.

The Mock Interview asks questions one at a time. If the candidate gives a vague answer, PrepPilot can challenge it with a focused follow-up asking for the concrete example that is missing. The goal is to recreate one of the most valuable parts of a real interview: being pushed beyond the answer you prepared in your head.

Afterward, the Scored Debrief evaluates answers across four dimensions: structure, specificity, evidence, and relevance. Candidates can see what worked, what was missing, how an answer could be stronger, and why. Improved answers remain grounded in evidence rather than quietly adding accomplishments the candidate never supplied.

Finally, PrepPilot creates a one-page prep sheet containing the positioning, gap reframes, priority questions, answer reminders, and questions to ask the employer that are most useful immediately before the interview.

The complete journey is:

Understand the role → Prove the fit → Face the gaps → Practice the conversation → Improve the answers → Walk in prepared.

How we built it

PrepPilot is not a single chatbot behind an interview website. We designed it as an AI-native multi-agent system powered by Gemini, where specialized agents have distinct responsibilities, decisions, and quality boundaries.

The Analyst interprets the job posting, separates requirements, searches the resume for supporting evidence, and determines whether each requirement is strongly evidenced, partially evidenced, or not evidenced.

The Strategist converts that evidence map into a personalized preparation experience. It creates twelve questions and determines which aspects of the candidate's background deserve the most preparation.

The Quality Critic provides an independent review layer. It evaluates generated preparation against a quality rubric and can block material before it reaches the candidate. This deliberately separates generation from judgment.

The Interviewer conducts the mock interview and makes a focused decision after an answer: move forward or probe for a more specific example.

The Evaluator reviews the candidate's responses after the mock interview, scores them across structure, specificity, evidence, and relevance, and provides grounded coaching on the answers worth improving.

We intentionally did not delegate every decision to an LLM. Gemini handles tasks that benefit from semantic understanding and judgment, while deterministic software handles rules that need predictable enforcement. These include structured-output validation, quote verification, unsupported-claim safeguards, the one-follow-up limit, session state, access control, and other application constraints.

That hybrid architecture became central to PrepPilot. AI reasons where reasoning adds value. Software constrains where consistency matters.

We also built an operational agent ledger so executions can be inspected rather than treating the AI pipeline as a black box. It lets us understand which agent ran, what operational decision it made, whether quality checks passed, latency, and cost, while being designed not to store resumes, job postings, interview answers, names, or model chain-of-thought in the operational record.

Challenges we ran into

One of the biggest lessons from PrepPilot was that many of the hardest AI problems were not solved by writing a better prompt.

Early in development, the Evaluator was required by its output contract to identify a minimum number of weaknesses. That seemed sensible until we tested a genuinely strong interview answer. The prompt told Gemini not to manufacture criticism, while the schema effectively required criticism to exist. We changed the contract so a strong answer could simply be recognized as strong.

We found another subtle problem in evidence verification. A normalization layer designed to make generated copy more readable was also modifying exact resume quotations. A small formatting change could make legitimate evidence fail verbatim verification and cause the system to incorrectly report that evidence was missing. We separated evidence integrity from presentation formatting.

Infrastructure created its own challenges. Model capacity failures and quota limits initially looked like content failures. We separated transport retries from content retries and introduced appropriate backoff so temporary infrastructure problems would not be mistaken for bad reasoning.

We also encountered a session persistence bug that appeared on Windows but had been hidden by Linux development behavior. A hardcoded temporary path and route-module memory isolation combined to produce a 404 after analysis. Fixing it reinforced a broader lesson: passing isolated tests is not enough. The complete user journey has to work in the environment the user actually runs.

Performance was another challenge. A multi-agent architecture provides useful specialization and independent review, but sequential model calls can make the product feel slow. We introduced caching, deterministic short-circuits, controlled concurrency, and clearer progress feedback. For example, a follow-up limit can be enforced in code without asking a model to make another unnecessary call.

Even repository management taught us something. At one stage, tested improvements existed in an isolated development environment but had not reached the Windows repository actually serving the application. We responded by introducing explicit parity verification rather than assuming that successful work in one environment meant the user was running the same code.

Each failure pushed PrepPilot toward the same engineering principle: trustworthy AI requires much more than a capable model.

Accomplishments that we're proud of

We are most proud that PrepPilot evolved from an interview-question idea into a working end-to-end AI preparation system.

A candidate can move from a raw resume and job posting to an evidence-backed Readiness Brief, personalized Interview Studio, interactive mock interview, scored debrief, and printable interview-day preparation sheet within one connected experience.

We are particularly proud of the evidence-first architecture. Career AI has a unique responsibility because a polished hallucination can become something a person repeats in a real interview. PrepPilot is designed to resist that temptation. If evidence is missing, the product can surface the gap. If a candidate supplies additional facts during practice, those facts can be used without falsely presenting them as resume evidence. If a generated claim cannot be grounded, the system should not turn it into part of the candidate's history.

We tested this behavior against difficult scenarios. PrepPilot was tested with strong answers that did not deserve artificial criticism, vague answers lacking specificity, candidates missing required experience, new facts introduced during the interview, and attempts embedded in candidate text to manipulate the evaluation. The Quality Critic was also tested against fabricated executive experience, unsupported budget ownership, and invented credentials.

We are equally proud of the multi-agent workflow itself. The system does not simply generate content and trust its own output. Generation, interviewing, evaluation, and quality control have separate responsibilities. The Quality Critic can reject material created by another agent before the candidate sees it.

We also designed PrepPilot with a business model rather than treating monetization as an afterthought. The Readiness Brief is the free entry point. The complete Interview Studio is designed as a $19 per-interview purchase, avoiding another recurring subscription for a need that becomes particularly intense around a specific event. For prototype evaluation, full access is complimentary so the complete product can be experienced without payment friction, while the commercial Stripe architecture remains preserved.

Most importantly, we built something that does not need to pretend AI knows the future. PrepPilot does not promise that someone will receive an offer, predict an employer's exact questions, or claim knowledge of proprietary hiring systems. It focuses on something we can actually help improve: preparation.

What we learned

Our biggest lesson was that an AI-native product is not defined by how many times it calls a model. It is defined by how intelligently AI is incorporated into the operating model of the product.

Different tasks require different kinds of intelligence. Extracting meaning from a job posting benefits from semantic reasoning. Deciding whether an answer needs a specific example benefits from contextual judgment. Checking whether an exact quotation exists in a resume should be deterministic. Enforcing a one-follow-up maximum should be deterministic. Quality control benefits from an independent reviewer.

Learning where not to use AI became as important as learning where to use it.

We also learned that good AI systems need permission to say less. An evaluator should be allowed to say an answer is already strong. An analyst should be allowed to say no evidence exists. A coach should be allowed to tell someone that they genuinely lack a qualification. Forcing AI to always produce something impressive can make the product less trustworthy.

Another lesson was that transparency and observability are different. Developers need operational evidence about agent executions, quality decisions, failures, latency, and cost. Candidates need a simple explanation of how PrepPilot reached the guidance they see. Designing those as separate experiences makes transparency useful without overwhelming users with engineering details.

Finally, we learned that personalization becomes much more meaningful when it is connected to a concrete objective. A resume alone does not provide enough context. A job posting alone does not provide enough context. An interview date alone does not provide enough context. Together, they turn a generic career problem into a specific preparation problem that AI can help solve.

What's next for PrepPilot

The next milestone is not simply adding more AI features. It is proving that PrepPilot creates value for real candidates.

Our initial business model is deliberately simple: the Readiness Brief is free, while the complete Interview Studio is designed as a $19 one-time purchase for a specific interview. This aligns the purchase with the moment when preparation has the highest immediate value and avoids forcing candidates into another subscription.

The next phase is to put PrepPilot in the hands of real candidates and measure the behaviors that matter. We want to understand whether candidates become more specific after a mock interview, whether they improve their use of evidence, which types of follow-up produce the greatest improvement, which parts of the preparation experience they return to before an interview, and ultimately which parts they value enough to pay for.

Over time, PrepPilot can become increasingly adaptive. A candidate who consistently struggles with specificity should receive a different preparation path from someone whose main challenge is structure or evidence. Multiple practice rounds could focus progressively on the dimensions where that individual needs the most improvement.

The opportunity can also extend beyond individual candidates. The same evidence-first preparation model could support university career centers, workforce development programs, career coaches, campus recruiting preparation, and people navigating career transitions. Those are opportunities to validate, not traction we claim to have today.

The larger vision is a preparation system that learns how to help someone become better at communicating their own experience without replacing their voice. We do not want candidates memorizing AI-written scripts. We want them to understand their stories well enough that they no longer need one.

The interview is still theirs. The experience is still theirs. The accomplishments are still theirs.

PrepPilot helps them prepare to communicate those things when the moment matters.

PrepPilot. Rehearse the interview before it happens.