Before Yes

Before a child gets an app, game or channel, a parent gets a verdict for that child's age, with every concern linked to its source.

Comment

Inspiration

I am a father of two, aged 9 and 7. I work deep in technology, and I still find it hard to know what my kids play. My wife and relatives find it much harder. Protecting our children in this day and age is important. A healthy child needs boundaries. Getting hooked on the wrong app or game can give a child trauma and open them to abuse.

News shows why parents worry. NBC Chicago, 19 Dec 2022, reports the FTC's record Epic Games settlement over Fortnite. Status: settled. FOX 4, 6 Nov 2025, reports Texas's lawsuit against Roblox over child safety. Status: an allegation, not a finding. The Associated Press via WTOP, 25 Mar 2026, reports a New Mexico jury found that Meta violated New Mexico's consumer protection law by harming children's mental health and safety, and ordered $375 million. Meta says it will appeal.

A parent like me gets asked "can I have Roblox?" several times a week, each time about a different title. The answers are spread across a privacy policy nobody reads, the terms of service, hundreds of reviews, parent forums and the occasional regulator's fine. A chatbot gives a fluent summary of a few of those pages, differently each time, and does not say where any claim came from.

Before Yes reads the whole public record for one title and one child's age. It returns a verdict, and each concern links to the quote it rests on.

What it does

  1. The parent enters a title and the child's age, from 3 to 17.
  2. The gather step collects the App Store listing and its reviews, the publisher's privacy policy and terms, Reddit threads, a parent forum, news and regulator coverage, rating pages, and the title's own parental-controls help pages.
  3. Nemotron-3 Nano reads every source in chunks. It returns findings, each with a verbatim quote and the source it came from. A finding whose quote does not appear in its source is dropped before the judge sees it.
  4. Nemotron-3 Ultra grades six areas against a fixed checklist: contact with strangers, purchases and spending, privacy and data, age restrictions, content, and developer history. It also writes a plain-language reason for each standing grade.
  5. Code checks the grades and computes the verdict. The verdict is one of yes, yes with settings, no or unclear. Missing evidence never reads as yes.

The card shows the verdict, the six areas, the settings to turn on, each concern with its quotes, and what was read and when. Three demo cards open from the home page without spending anything: Roblox (9) is no, Minecraft (9) is yes with settings, and Khan Academy Kids (5) is yes.

How Nemotron and Token Factory are used

Both halves of the platform rule run on Nebius Token Factory. The check calls no other model host.

  • Nemotron-3 Nano (30B-A3B) reads. Each source is split into chunks, and each chunk becomes quoted findings. Thinking is off for this role, because Token Factory bills thinking as output tokens. Nano does the high-volume work, so every source is read in full.
  • Nemotron-3 Ultra (550B-A55B) judges. The first pass grades all six areas with thinking off and repetition penalty 1.1. Where an area's findings pull both ways, a risk and a control for example, a second pass runs with thinking on, for those areas only, at temperature 0.6, top_p 0.95 and repetition penalty 1.05.
  • Why those sampling settings. Ultra looped at temperature 0. Thinking-on conflict passes repeated a line until their token guard, and thinking-off first passes repeated finding ids until their cap. The sampling settings above stopped both. The client sends temperature 0 by default, and the judge overrides it per call.
  • Why Ultra. We tried Nemotron-3.5 Lightning, Nemotron-3.5 Super-VL and Ultra on the same evidence. Ultra was the first to reach the expected verdict on Khan Academy Kids without a rule override.
  • Quote verification. Before the judge sees anything, every finding's quote must appear in the source it names, after normalisation. That is a string match in code. A finding that fails it is dropped.
  • Checks the model cannot skip. Citation pruning removes grades that cite nothing valid. Grounding requires a named setting to appear in a cited quote. Entailment asks Ultra, with thinking off, whether the cited quotes meet the definition and whether the reason says only what they support. A deterministic rule gate in Go can veto, cap, confirm or raise a grade. Every action is listed on the card under "Rule checks". Then checklist.Decide computes the verdict from the six grades.
  • Measured cost. Model spend per fresh check averages $0.106976 over nine checks: Roblox $0.116922, Minecraft $0.110137, Khan Academy Kids $0.093868. Nano is priced at $0.06 in and $0.24 out per million tokens. Ultra's price is an assumption of $1 in and $3 out, because its model page did not show one.

Token Factory's OpenAI-style API made the judge switch a settings change: the host, model id, prices and rate limits are in before-yes.toml. Thinking and sampling are per-call switches, so the reader and judge differ in one client.

How we built it

  • Server: Go, built on Keel v0.7.3. Keel supplies the job runner (a check outlives its request), progress streaming with replay, the cost meter, the cache, the gate, the feature flag, config, ids, error format and SQLite storage.
  • Web app: Svelte 5 on Vite, using Chaaya 0.4.2 for the job stream and the theme and accessibility tokens, and Bits UI for dialogs and tooltips.
  • Sources: one Tavily client for search and extract, and an Apple review and listing fetcher. Each check runs eight Tavily searches and a listing read.
  • Spend limits: a $20 model ceiling from the $25 Token Factory credit, 900 Tavily credits a month, and 5 fresh checks per UTC day. Cached cards and evidence cost nothing. Visitors may use their own keys, which are held for that request only.
  • Deployment: one container image with a distroless runtime, behind Traefik on a Hetzner host, with Cloudflare in front and a nightly backup of the databases.
  • Tests: the Go gate runs gofmt, go vet and go test -race, and the web app runs lint, type check and browser tests. An offline replay (go run ./tools/eval -models replay) reruns the recorded model replies and needs no keys.

Challenges

  • The Apple review feed went dark during the build. It returned HTTP 200 with no entries, the same answer as an app nobody reviewed. We added a fallback that reads reviews from the App Store web page through Tavily, which gives three or four reviews per title with no rating or date. The feed was back for the final live check, which read 500 reviews, so the pipeline tries the feed first and keeps the fallback.
  • Ultra looped at temperature 0. Sampling settings fixed it. See the model section above.
  • Verdicts move between gathers. On the three demo titles, Roblox is no on all three recorded gathers. Minecraft is yes with settings on one and unclear on two. Khan Academy Kids is yes on two and unclear on one. We recorded the movement rather than tuning it away.
  • Many raw findings do not survive the quote check. On the offline replay, 36.7% of Roblox's raw findings, 35.7% of Minecraft's and 27.7% of Khan Academy Kids' were dropped, mostly because the quote is not in the source text.
  • A matching quote does not prove a sound reason. The quote check confirms that the words exist. Entailment and the rule gate exist because a grade's reason can say more than its quotes do.

Accomplishments

  • Every concern on a card links to a verbatim quote and its source. The quote is checked by string match in code, not by the model's word.
  • On the offline replay of the recorded evidence, all three verdicts match the expected results, and 12 of 18 area grades do. The replay also shows the limit: only 3 of 16 decisive quotes were recovered. We report both.
  • A check costs about 11 cents of model calls, measured on recordings at an assumed Ultra price.
  • The hosted app ran a live check on a title outside the demo set, Duolingo ABC for a 6-year-old, on Token Factory and Tavily. It cost $0.078 of model calls and 10 Tavily credits, and returned unclear.
  • Missing evidence never reads as yes. The card says "unclear" and names what was missing.

What we learned

  • The verdict depends on how much of the evidence the model actually reads, so the pipeline reads every chunk and does not sample.
  • A model's grade is not evidence. Code checks the quotes, the citations, the grounding and the verdict rule. The entailment step is the one check that is itself a model call, and it can be wrong too: on one Minecraft gather it accepted reasons its quotes did not support, and it once refused an earned flag until its instruction was clarified.
  • Thinking is a cost decision made per call. On one Nano prompt, thinking-on used 183 output tokens and thinking-off used 4, about 45 times the output tokens.
  • "Unclear" is a useful answer. A parent is better served by a verdict that says what is missing than by a confident guess.

What's next

  • Show a checked safer alternative on the demo cards. The alternatives step is built and tested, and it runs one Tavily search and a full check on up to three candidates. No demo card shows one yet, because that needs live checks.
  • Make verdicts stable across gathers, and check whether the quote capture for reviews can improve.
  • Read Google Play. It has no official API, so it is out of scope for now.

Built With

Share this project:

Updates

Submission history