What we built and why

Families ask one question: where do I stand? Today the answer is a rumor, a ranking, or a percentage nobody can check. We built a screen that answers it honestly. It opens with the student's strengths in their own words, shows each college as a distance from the student's score, shows who the student resembles among past applicants, and ends with a plan to take home. No odds, and every label can be traced to its source. How it handles accuracy and uncertainty

Code sets every label and number from published federal data; a model only reads the essay. All 60 labels on a hand-worked answer sheet match. "Not sure" is a normal answer: "can't say" when a college publishes no score range, "too few cases to say anything yet" under 15 cases, "this one is close" when 30 points would change a label. Past cases are shown as counts beside the official admit rate, because we measured that people who post their results are admitted more than three times as often as real applicants at the most selective colleges. One switch, Show the working, opens every source, rule and check. Tools used

Nicrron: every AI call. Two models from two different companies read each essay, so every answer comes with a built-in second opinion; Nicrron also powers essay search and reviewed our code. PRISM by Block Convey: a guardrail that blocks essays carrying hidden instructions, a knowledge base every college statement is checked against, and a record of every run. Python (standard library only), plain HTML, CSS and JavaScript. Data: US College Scorecard, Common Data Set, public self-reported results. Team names and roles: yours to fill in; I have not assumed who did what.

The project story, remade Updated on the dashboard: https://dashboard.alexmong.com/get-hired-1010/materials/plan/devpost_story.md?key=k-5d5e943c588d

What changed: “Many hands, one product” now lists four pieces including essay search; the rebuilt screen has five steps; “Under the hood” adds essay search, essay text in closeness, and 357 tests.

The question in the room

Picture a counselor's office in October. A seventeen-year-old sits down with a list of dream schools. A parent is in the next chair, quietly worried about money. They have one question: where do I stand?

Today the answers come from rankings, from rumor, and from tools that hand back a single number: "you have a 23% chance." Nobody in the room can check that number. Nobody can say where it came from.

That was our starting point. The challenge asked for a copilot that finds similar past applicants and builds a list of reach, target and likely schools "grounded in real outcomes." The theme of the day was AI you can trust.

What we found when we went looking

We started by hunting for real outcomes, and found a problem hiding in plain sight.

The public data on who got in where comes from people who chose to post their results online. And people who post are not typical. We measured it. At the most selective colleges, posters said they were admitted more than three times as often as applicants really are.

So a tool that looked up "students like you" and reported how many got in would be telling a hopeful family something untrue, with a confident face. That is the opposite of trust.

That finding changed what we built.

What we decided to build instead

We made three promises.

We will never invent a personal chance of admission. There is no percentage anywhere on the screen. If a family asks for one, the tool explains why no one can honestly give it, and points to what it can show.

Everything can be checked. Each school's label comes from numbers the college itself publishes, using a rule simple enough to read aloud. Real past cases are shown as plain counts: this many admitted, this many not. One switch, Show the working, opens up every source and every rule.

We say what we don't know. When there are too few cases to mean anything, the screen says "too few cases to say anything yet." When a college publishes no score range, it says "can't say." When cases come from that skewed public pool, it says so right beside them.

Many hands, one product

We split up and built several things at once.

  • The meeting screen and the rule that labels each school.
  • Closeness: a way to show who a student resembles at each college, part by part, next to the nearest students who got in and the nearest who did not.
  • An evidence store: a careful home for applicant records that remembers where each fact came from, and admits how much of a profile it could actually compare.
  • Essay search: paste a draft, and it finds essays on similar subjects from a small bank of openly licensed examples. It compares what essays are about. It never judges how good one is.

With a little over an hour left, we pulled them together. It was not tidy. One set of views had been built on a version of the screen we had since thrown away, and had to be carried across by hand. Then teammates merged again on top of that. But by the end, one page told one story, and pressing any college showed the pieces working as one: the label, who you resemble, and the evidence behind it.

The moment it wasn't good enough

By early afternoon the labels were right. Every one matched an answer sheet worked out by hand, separately from the code: 60 out of 60.

Then we asked reviewers to look at the screen the way a stranger would, from pictures alone. Their verdict stung. The first thing a family saw was "0 Likely" next to a yellow warning. The student appeared only as a test score. One reviewer, imagining the seventeen-year-old, wrote that "Reach" read as I'm not good enough.

We had built something accurate and cold. These are stressed people with dreams, and we had handed them a dashboard.

So we rebuilt it as a story:

  1. Who you are. It opens with the student's strengths, in their own words from their essay.
  2. Where you stand. One picture, like a trail map: you are here, and each college is a place ahead, with the distance written on it.
  3. What is in your hands. Schools that would balance the list, a slider that shows what a different score would change, and what each college says it looks for.
  4. Your plan. Five things to do this month, printed for the family to take home.
  5. Essay search. Essays on similar subjects, to spark ideas for the next draft.

"Reach" is now drawn as ambition, not alarm. Hard news is said kindly, and is always followed by something the student can do.

What else went wrong

  • We missed privacy holes, twice. We asked outside reviewers to attack our code, and they slipped a dozen kinds of personal detail past our filter. Later, a different AI model we asked for a second opinion found one more: street addresses written in lower case. We fixed each one. Now, if the privacy check cannot run, the essay is simply not read.
  • We mislabeled our own practice data. Our made-up school history briefly showed up as "verified by a counselor." It is now marked as invented wherever it appears.
  • We burned through a daily allowance on one of our required tools by accident, and finished the day on what was left.

What the data taught us

Late in the day we built a tool to read thousands of those public posts properly, and looked at what they could honestly tell us. Three things stood out. All of them describe people who chose to post, not applicants in general.

  • The list matters more than anything on it. Among posters whose list had no safer choice at all, about one in five was admitted nowhere. Among those who had at least one, it was about one in fifty.
  • At the most selective colleges, the test score barely separates who gets in. A gap of more than 150 points moved the outcome only a little.
  • No essay topic "works." Whatever people said they wrote about, family, work, identity, a hobby, the results looked nearly the same.

Each one backed a choice we had already made: put list balance front and center, call the hardest colleges "a reach for everyone," and never let an essay move a label.

What we learned

  • Admitting what you don't know builds trust. "Too few cases to say" is a more believable sentence than any percentage.
  • Stay quiet when you're right, speak up when you're wrong. A "verified" badge on everything teaches people to stop reading it. Showing only the failures made the checks mean something.
  • Ask who each number is for. The family needs the picture. The counselor needs the reason, one tap away. The skeptic needs the working. Putting all three on one screen helped nobody.
  • Get someone else to check your work. The reviews that hurt most, and helped most, came from outside the group that built each piece.

What's next

The honest source of "students like you" is a school's own past students. So each college on a plan carries a status, and when a result comes in, it goes back into that school's cases, with no names, ready for next year's class. Real, verified cases are exactly what Nerd Apply and its counselors hold. That is where this should go.


Under the hood

For readers who want the technical detail.

The rule. Code sets every label. A model never does. For a student's score $s$ and a college with admit rate $a$ and a middle score range from $P_{25}$ to $P_{75}$:

$$ \text{label}= \begin{cases} \text{reach for everyone} & a < 15\%\ \text{can't say} & \text{no score, or no published range}\ \text{reach} & s < P_{25}\ \text{likely} & s \ge P_{75}\ \text{and}\ a \ge 40\%\ \text{target} & \text{otherwise} \end{cases} $$

Closeness. For each college, the distance between the student and past cases is measured part by part (score, major, background, home state, and, when a model key is set, essay and activity text), then compared between the group that was admitted and the group that was not. The result is a sentence such as "looks more like the people admitted here," never a forecast. We tested it by hiding each of about 400 past cases in turn and counting how the result came out.

Applicant evidence. A small database for applicant records that keeps the source and consent for each one, takes in messy files, and scores how alike two profiles are with stated weights. It also reports how much of a profile it could compare, so a thin match cannot rank first. Its scorer orders the example cases on screen.

Essay search. A draft is compared with a small reference bank: openly licensed essays, each credited to its source, plus a few invented samples. It uses text embeddings when a model key is set and plain word overlap otherwise, and says which. The student's draft has personal details hidden first and is never saved.

Data. 1,339 colleges from the US government's College Scorecard, including what families actually paid after grants. Each college's own published statement of what it weighs, for 24 colleges. About 2,250 public self-reported applicants. An invented school history, labelled as invented, standing in for a counselor's records.

Nicrron carries every AI call. Once personal details are hidden, Nicrron puts the essay in front of two models from two different companies at once: one draws out the themes and the student's own words, each note tied to a word-for-word quote; the other gives an independent second opinion on the intended major. One key opened the door to both, so every answer comes with a built-in second opinion, and another model steps in if one is unavailable. Nicrron also supplies the embeddings behind essay search, and a third model on Nicrron reviewed our own code and caught a privacy bug.

PRISM by Block Convey does three things: it screens each essay and blocks one that carries hidden instructions for the AI; it holds each college's statements so every claim can be confirmed against them (a false one we planted came back "not confirmed"); and it keeps a record of every step of every run.

Testing. 60 of 60 labels match a hand-worked answer sheet. 357 automated tests, including one that fails the build if the language of odds appears anywhere in a result.

Built with Python (standard library only) and a plain HTML, CSS and JavaScript screen. Parts were built in parallel, by people and by coding agents, from one shared agreement about the shape of the result.

Built With

  • anthrophic
  • api
  • nicrron.ai
  • prism
  • python
Share this project:

Updates

Submission history