StratCO

Inspiration

Software engineering has LeetCode and GitHub—objective, verifiable proof of skill that lets strong self-taught builders bypass resume screens. Finance, strategy, and consulting recruiting have the opposite problem: hiring still leans heavily on school prestige, while traditional take-home case studies have become almost impossible to trust now that anyone can paste a prompt into a chatbot.

We asked: What if business cases were a verifiable skill credential, and what if the benchmark wasn't an arbitrary rubric score, but whether your judgment actually beats an AI baseline on the exact same dataset?

That question became StratCO (StratCode)—a platform where candidates solve realistic small-business problems in a structured, three-stage workflow, get graded by a hybrid deterministic + rubric pipeline, and earn direct vouches from founders who review the top solutions.

What it does

StratCO gives solvers realistic small-business case briefs paired with raw CSV datasets across three challenges:

1. Brew & Co. — Diagnose why a neighborhood coffee shop's variable margin fell between quarters and decompose the change into mix, price, and unit-cost effects.
2. Crumb & Crust — Evaluate whether a bakery's $10\%$ across-the-board price increase actually raised variable profit and compute the break-even volume loss.
3. Pressed & Proper — Decide whether a fast-casual sandwich and juice shop should sign a second-location lease by modeling standalone vs. cannibalization-adjusted payback periods.

Every challenge is completed in a three-stage workspace:

• Stage 1: Issue Tree — Solvers build a nested, MECE (mutually exclusive, collectively exhaustive) outline up to 4 levels deep to structure the business problem.
• Stage 2: Numbers — Solvers compute 6 quantitative metrics from the CSV dataset (e.g., margin bridges, price elasticity break-even $\Delta V = 1 - \frac{M_{\text{before}}}{M_{\text{after}}}$, and cannibalization-adjusted payback).
• Stage 3: One-Page Memo — Solvers write a concise ($\le 300$ words), answer-first executive recommendation citing their Stage 2 numbers.

When a solver submits, StratCO computes a weighted score out of $100$:

$$\text{Total Score} = 3 \times S_{\text{tree}} + 3 \times S_{\text{numbers}} + 4 \times S_{\text{memo}}, \quad S_i \in [0, 10]$$

• Stage 2 is graded deterministically on the server against hidden ground-truth answers within strict absolute tolerances ($|x_{\text{submitted}} - x_{\text{true}}| \le \tau$).
• Stages 1 and 3 are graded server-side by Gemini against weighted, criterion-level rubrics and compared directly against a pregenerated AI baseline—what an unassisted LLM produces when given only the case brief and CSV.
• Founder Dashboard & Vouching — Founders see the top 3 graded submissions per challenge along with a two-sentence executive summary of each solver's recommendation and can vouch for candidates, attaching a verification badge and outcome note to the solver's profile and leaderboard entry.

How we built it

• Frontend & Framework: Built with Next.js 16 (App Router, TypeScript, React 19, Server Components, and Server Actions) and styled with Tailwind CSS v4 and shadcn/ui following an Apple-inspired minimalist design language (system typography, tabular numerals for financial data, hairline borders, and full light/dark mode support).
• Database & Security: Powered by Supabase (PostgreSQL + Auth). We enforced defense-in-depth at the Postgres layer:
  • Row-Level Security (RLS) on every table (profiles, challenges, challenge_answers, submissions, vouches).
  • Column-level privilege grants so authenticated clients can only update stage1, stage2, and stage3 on their own draft submissions and display_name on their profile—preventing users from self-promoting their role to founder or forging their own scores, status, or baseline_delta.
  • Isolated ground-truth answers: challenge_answers has zero client read policies and is only accessible via the server-only secret key.
• Hybrid Grading & Validation Pipeline: Implemented in /api/submissions/[id]/submit using Zod for strict input and LLM-output schema validation, and @google/genai for structured JSON rubric evaluation. Solver text is wrapped in explicit data delimiters (<solver_tree>, <solver_memo>) to defend against prompt injection, and Stage 3 cross-checks whether numbers cited in the memo match the solver's actual Stage 2 calculations.
• Data Visualization: Recharts renders side-by-side stage comparisons (You vs. AI Baseline) on the results view.

Challenges we ran into

• Preventing LLM grade inflation and calibrating rankings: Early prompts were too generous to vague, buzzword-heavy memos. We built an offline calibration harness (scripts/calibrate.ts) with hand-crafted weak, medium, and strong exemplar submissions for every challenge, iterating on criterion weights, temperature ($0.2$), and Stage 2 cross-check instructions until the grader reliably enforced strict monotonic ordering ($\text{weak} < \text{medium} < \text{strong}$) across Stage 1, Stage 3, and Total Score.
• Locking down answers, rubrics, and roles without trusting the client: In a BaaS setup, select * or permissive table grants can accidentally leak grading rubrics, AI baselines, or answer keys to the browser network tab. We revoked table-wide privileges, wrote explicit column-level GRANT SELECT and GRANT UPDATE statements in SQL migrations, banned select * across the codebase, and wrote an automated 11-check security test suite (scripts/rls-check.ts) that logs in as two adversarial test users and verifies every unauthorized read/write is blocked.
• Race conditions during grading: Double-clicking submit could trigger parallel Gemini calls. We implemented an atomic compare-and-swap claim (UPDATE submissions SET status = 'grading' WHERE id = $1 AND status = 'draft') that locks the submission before calling Gemini and cleanly rolls back to draft if an upstream API error occurs.

Accomplishments that we're proud of

• Beating the AI baseline as a meaningful signal: Instead of pretending candidates don't use AI, StratCO measures the exact delta ($\Delta_{\text{baseline}} = \text{Score}{\text{solver}} - \text{Score}{\text{AI}}$) between a candidate's structured reasoning and a zero-shot AI analyst.
• End-to-end verification & calibration tooling: Shipping not just the web app, but a full suite of CLI verification scripts—baseline generation (generate-baseline.ts), rubric calibration (calibrate.ts), demo population (seed-demo.ts), and adversarial RLS verification (rls-check.ts).
• Clean, distraction-free UX: Designing a workspace where an interactive tree builder, a live CSV data table, numeric inputs, and a markdown memo editor fit together with zero visual clutter.

What we learned

• Hybrid evaluation beats pure-LLM or pure-multiple-choice grading: Pure multiple-choice misses strategic synthesis, while pure LLM grading can be gamed by confident prose that gets the unit economics wrong. Feeding deterministic Stage 2 accuracy results directly into the Stage 3 rubric evaluator gave us the best of both worlds—memos that hallucinate numbers or ignore calculation errors are automatically penalized.
• Postgres column-level security is underrated: Combining RLS row policies with column-level REVOKE / GRANT let us use direct Supabase client queries for fast draft autosaving while keeping grading fields, rubrics, and role elevation strictly server-controlled.

What's next for StratCO

• Live founder-sourced micro-cases: Partnering with small businesses and startups to turn anonymized operational datasets into live challenges where founders recruit directly from the top 3 leaderboard slots.
• Interactive sensitivity modeling: Expanding Stage 2 from point estimates into multi-scenario sensitivity tables and unit-economics charts inside the workspace.
• Peer & human-calibrated benchmarks: Combining LLM rubric scores with blinded peer and founder reviews to continuously calibrate rubric weights over time.

Built With

• claude
• gemini
• next.js
• supabase
• tailwind
• vercel