Scout - New Age Interview Pattern

Scout is made to replace traditional style Leetcode Interviews, and replace them with real software engineering challenges, which reflects their day to day job scenarios

Comment

Scout - Devpost Submission

Project name

Scout - Repository Interviews

Elevator pitch

Scout replaces LeetCode-style coding rounds with browser-based repository interviews, benchmarks, and evidence-based scoring.

Built with

Next.js, React, TypeScript, Node.js, SQLite, Docker, code-server, Python, C++, OpenAI Codex, GPT-5.6, Vercel AI SDK

Try it out

About the project

Inspiration

Most engineering interviews still test isolated puzzles in a blank editor. That makes it easy to optimize for memorized patterns or a single AI-generated answer, while missing the work engineers actually do: understand an unfamiliar system, find the important bottleneck, preserve behavior, test the change, and explain the trade-off.

Scout turns that work into the interview.

What it does

Scout gives candidates a realistic, browser-based VS Code workspace backed by an isolated Docker container. Each session starts from a production-shaped repository, engineering ticket, tests, benchmark, and intentional TODO seams.

The prototype includes nine Python, TypeScript, and C++ interview repositories with 45 measurable optimization opportunities across routing, caching, scheduling, matching, payments, recommendations, search, log processing, and risk allocation.

Candidates are not expected to finish every mission. The scorecard rewards meaningful progress: correctness, benchmark improvement, progress toward a golden reference, and hidden-test confidence. Two or three strong improvements can be a strong interview result.

Interviewers can view elapsed time, the live candidate VS Code workspace, submissions, test evidence, benchmark deltas, mission scores, and the data-structure concepts demonstrated.

How we built it

Scout uses Next.js, React, TypeScript, SQLite, Docker, and code-server. Starting a session copies a candidate repository, applies a reproducible workload variant, launches the matching language container, and opens VS Code in the browser. The terminal and hidden grader use the same workspace, so candidate tests, benchmarks, and the submitted code run in one toolchain.

The grader runs hidden invariants and benchmarks, parses a structured report, and generates the interviewer scorecard. Each pack has a candidate version, golden reference, visible tests, hidden tests, and calibrated mission rubric.

The AI interview guide is honestly shown as Coming soon. This demo focuses on the working repository workflow and deterministic evidence rather than presenting an unfinished assistant as a completed feature.

How we used OpenAI Codex and GPT-5.6

I used OpenAI Codex with GPT-5.6 as a development collaborator. It helped decompose the work, iterate on the Next.js and Docker workflow, structure challenge packs, improve the UI, troubleshoot grading and build issues, and refine documentation and tests.

I set the interview philosophy, designed the repository missions and scoring rubric, reviewed the implementation, ran the validation checks, and made the final product decisions. GPT-5.6 is not Scout's runtime interviewer or grader: current scores are deterministic and come from tests and benchmarks.

Challenges and learnings

The main challenge was balancing realism with reproducible evaluation. Scout uses persisted replay variants to change workload shape and ticket framing while keeping the public API and rubric comparable. We also avoided an all-or-nothing grade because real candidates will not optimize every path in one interview.

We learned that the best interview evidence is not a pass/fail label. It is a clear record of what the candidate changed, what they tested, how performance moved, and what opportunities remained.

What's next

Next steps are authentication, protected grader storage, read-only live observation, more calibrated repositories, and a constrained AI guide for repository navigation rather than solution generation.

Built With

Share this project:

Updates