BinRules

Which bin? Only what your council's own list says.

Comment

Summary

BinRules answers "which bin does this go in?" using only a city's official sorting list. It shows the exact entry it used, with the council's own note and a link to the source, and it says "not covered" when the list does not name the item.

On Brisbane's official list, the same Nemotron model gets 100% of official item names right with the list behind it and 66% without it. For items the list never mentions, BinRules refused 94% of the time and the model alone refused 4.5%.

Live demo: https://binrules.vercel.app Code (MIT): https://github.com/StuartYo/binrules Full results and every failure: https://github.com/StuartYo/binrules/blob/main/eval/RESULTS.md

Theme track: the waste problem and how I would measure it

The problem I targeted is wrong-bin sorting at the kerb. Sorting rules are set city by city, residents guess, and a wrong item in the recycling bin contaminates the load: where communities measure it, contamination at the plant averages about 17% by weight, and only about a third of US communities know their own number (Recycling Partnership, 2020 Curbside Report, as reported by Waste360 and Recycling Today). BinRules removes the guess by answering only from the council's official list, citing the entry, and saying "not covered" when the list is silent. In this project I measured one thing: whether an answer is backed by the council's own entry. On 195 official item names it was right 100% of the time against 66% for the same model without the list, and it refused 94% of 67 items the list does not cover against 4.5%. To measure whether it reduces waste in a real city, I would hand-sort sampled loads from a pilot neighbourhood and a matched control for a few weeks before launch and again after, then compare the change in contamination by weight. A cheaper early signal is the log of "not covered" questions, which lists the items residents ask about that the council's own list never mentions.

The waste problem I targeted

A pizza box goes in the yellow bin in Brisbane. In other cities the same box depends on whether it is food-soiled, and the rule is often set by the contract with the local processing plant. A general model answers this kind of question with one confident bin. I tested what that is worth. Asked about Brisbane's official item names, the same model alone picked a bin the council does not list for 34% of them (66 of 195). Asked about 40 items that Brisbane's list does not name, it answered 37 of them. Nothing in its answer says which rule it used or whether the council has one.

What it does

  • You type an item the way a person would say it ("greasy box my pizza came in", "icy pole sticks").
  • If the city's official list has an entry for that item, you get the bin or bins, the council's own note, the entry id, and links to the source record.
  • If an item has condition variants and your words do not pick one, you get all of them. Type "shoes" and you get "Shoes (poor condition)" and "Shoes (good condition)", each with its own bins.
  • If no entry names the item, you get "not covered", the nearest entries marked as not a match, and a link to the council. It does not fall back to a general rule or to the nearest material.
  • A second panel asks the same model with no list, so you can see the difference on your own items.

The live site has one city, Brisbane (294 entries, CC BY 4.0, credited to Brisbane City Council). The code takes any city whose official list is normalised into one JSON file.

How I built it

The model only maps words to entries. It never sees the bins and never writes the answer.

  1. Nemotron 3 Super on Nebius Token Factory rewrites the resident's words into plain item names.
  2. Code searches the official entry names with a word index (BM25 with typo tolerance) and keeps the closest 14.
  3. The model is shown those entry names in two different orders and says which one names the same item, or none. A part, a container, an item made of the same material, or a broader category does not count.
  4. Code requires that both orders agree, that the pick shares a word with what the resident wrote, and that a separate yes/no question about the final pick comes back "yes". Any failure ends in "not covered".
  5. Code reads the bins and the note from the official entry.

Because the model's only output is an entry number, an instruction hidden in the item text cannot produce a bin. There is a test for it. The site is Next.js on Vercel. The public demo has per-IP and daily caps, saved example answers that cost nothing, and a labelled word-for-word fallback when the day's model budget is spent. A live check costs about $0.0006 with the comparison panel on.

How I measured it

Same model on both sides, temperature 0. The "model alone" side gets the city's bin labels with a short description and may answer UNSURE, so it is not punished for being unable to refuse. I wrote the test questions and froze them (hashed in the repo) before tuning. Tuning used a separate dev split. Scoring is strict for BinRules (it must cite an entry I accepted) and generous for the model (any bin from the official entry's options counts).

Brisbane test split:

  • Official item names right (n = 195): BinRules 100%, model alone 66.2%
  • Messy phrasing right (n = 136): BinRules 86.0%, model alone 67.6%; wrong: 1.5% vs 32.4%
  • Items the list does not cover, refused (n = 67): BinRules 94.0%, model alone 4.5%

Messy phrasing is slang, typos, brand names and long descriptions that I wrote. The 67 uncovered items are 40 items the list does not name plus 27 entries that I deleted from the knowledge base before asking their own names. In that deleted-entry test the model alone still landed on a bin the full list allows 20 times out of 27, but it cannot tell you which answers those are.

All numbers come from one results file, and a test fails if the README or the report drift from it. The plan for measuring the effect in a real city is in the theme-track paragraph above.

Challenges

  • The small model was unstable at picking an entry from a list: the same question gave different picks in different orderings. Asking for agreement across two orderings and a separate yes/no check fixed most of it. The cost is that BinRules declines about one in eight messy questions.
  • Two bugs found after test runs forced two re-runs of the whole test split. The word "can" was on my stop-word list, so "Oil can" collapsed to "Oil". Later, the model's own rewrites were allowed to choose between condition variants, which made an item that a list splits by condition answer with one variant only (I saw it on the second city's pizza-box entry, since dropped, and fixed it before the final runs; Brisbane's list splits other items, such as shoes in poor or good condition). Both first-run numbers are kept in the report.
  • Some of my own "not covered" labels are arguable, because the list has broad entries such as "Household appliances". I did not relabel after seeing results.
  • I first built a second city, Toronto, and dropped it before submitting. The City's open-data page lists that dataset's licence as "not specified", so I removed its data, questions and results from the current repo and the site instead of publishing them. Earlier commits in the public git history still contain them.

Limits

  • It will not answer class-level questions. "Toaster" gets "not covered" plus the nearest entries when the list only says "Household appliances". That is deliberate and it costs recall.
  • One city on the live site. Adding a city means normalising its official list.
  • Rules change. Each answer shows the dataset's last-modified date.
  • I measured accuracy against the official list, not the behaviour of real residents or contamination at a plant.
  • I labelled the out-of-scope questions myself and nobody else checked them.
  • The hosted model is not bit-for-bit deterministic, so re-runs differ by about two points.
  • The demo's caps are per server instance. If the page shows "word-for-word lookup only", the day's budget is spent and the full pipeline is back after 00:00 UTC.

What I would do next

Add cities whose official lists are published as data under a clear licence, and send a council the "not covered" log as a list of gaps in its own rules. Run a bin-audit pilot with one council to see whether contamination actually moves. Accept a photo as input by turning it into item text first, so the answer still comes from the entry and not from the picture.

Try it

  1. Open https://binrules.vercel.app.
  2. Click "tea bags". The official entry says Red bin and quotes the council's note. The model alone says the garden organics (green) bin, which is not what the council lists for this item.
  3. Click "kitchen sponge". Brisbane's list has no entry, so you get "not covered". The model alone answers anyway.
  4. Click "shoes". The list splits shoes by condition, so you get both official entries, one per condition.
  5. Type "icy pole sticks". The Australian slang resolves to the entry "Paddle pop sticks (wooden)".
  6. Open the "How this was checked" line under any answer, then open https://binrules.vercel.app/eval for the tables.

Built with

Nemotron 3 Super on Nebius Token Factory, Next.js, TypeScript, Vercel, Vitest. Data: Brisbane City Council "What goes in my bin" open data (CC BY 4.0, reformatted, wording unchanged). BinRules is an independent student project and is not affiliated with Brisbane City Council.

Method note

Freezing test questions before tuning and comparing against the same model without the system comes from my earlier hackathon entries. No code is reused, apart from the daily-cap module, which is adapted from my Fair Shift project. Evaluation and development cost about $1.32 of a $2.00 project cap.

Built With

Share this project:

Updates

Submission history