ShelfSkills

Inspiration

Grocery products arrive as a short name, a description, and an ingredient list that does not always say the thing the label cares about. Each attribute has a closed list of allowed strings, and a value counts only when it matches exactly. A near miss is a miss. A confident wrong string is worse than silence, because a bad guess is charged as both a false positive and a false negative, while an empty answer costs only the miss. That scoring rule is what we built around.

What it does

ShelfSkills reads one product and returns exact attribute strings. Each attribute has its own learned skill. The skill emits a value when the text supports that exact string, and leaves the field empty when it does not. Unannotated attributes are ignored.

Quality is exact-match micro-F1:

$$ Q = \frac{2\,TP}{2\,TP + FP + FN} $$

On the experiment validation split, 202 products the weights never saw, an empty agent scores 0 and a literal copy of allowed strings scores 0.4662. ShelfSkills scores 0.8909 there, with false positives falling from 310 to 75. On the full hold-out, Q is 0.854712 across all 415 products. The experiment number is the learning evidence. The hold-out number is the performance score.

How we built it

evolve fits one averaged perceptron per attribute on tokens from the name, the description, and the ingredients. A product is in the validation split when the first eight hexadecimal characters of MD5(product id), read as an integer, leave a remainder of 0 on division by 5. That split chooses the text view, the margin required before a value is emitted, the training pass, and which candidate readings to keep.

The detectors and the wording of those readings were written by hand. The experiment labels decide the weights, and they decide which readings stay. A mined rule or a name phrase is kept only when validation micro-F1 rises. Diet, seed, and garlic also need at least nine in ten of their training rows. The other readings stay only when every training row that shows the cue matches and validation micro-F1 does not fall. After that choice, the same packs are refit on all 965 experiment products. That refit is the saved skill bank. predict loads it and does not learn. Foundation-model parameters are not trained.

Coding wise we used Cursor for that, and for the hold-out tests, and used Grok Agents to form a council and think of ways to improve our model, and took the best pieces from each ones.

Challenges we ran into

The text often fails to say the answer. A second grain is sometimes the single label Multiple and sometimes the grains named one by one. Forcing Multiple whenever a second grain appeared lowered an earlier result from 0.8660 to 0.8638. Corn syrup and corn starch are sweeteners, not a corn grain, so the word corn by itself is a trap. A sweetener is not always written as sugar or as corn syrup. A fruit piece is sometimes mixed through the cup and sometimes only a flavor. A culture name can belong to a traditional yogurt.

Readings that looked strong also failed when we applied them too widely. A nearest-example memory scored about 0.70. Dropping the product name fell to 0.838. Applying every reliable name phrase at once fell to about 0.79. A reading that is often right still loses when it fires on every product.

Accomplishments that we're proud of

The agent improves by updating reusable skills, not foundation-model weights: the perceptron, the text view, the margin, the epoch, and the keep-or-drop decision for each reading. On the same 202 products and the same metric, that moves the score from a literal copy at 0.4662 to learned skills at 0.8909. The hold-out pass covers every product, with validity 1.0 and no missing rows. The bank is frozen before that pass, and prediction does not learn.

What we learned

Silence is a legitimate answer. The literal copy looks reasonable because it echoes strings that are actually written down, and it still emits flavor copy, a grain named inside a sweetener, and marketing phrases. The gain came from learning when not to speak. We also learned to keep two numbers apart. The 0.8909 figure is the fair experiment comparison. The 0.854712 figure is the hold-out score. The refit on all 965 experiment products is the agent that predicts, and it is not the learning-evidence number.

What's next for ShelfSkills

The remaining misses are the cases where the package never names the attribute cleanly: a second grain that might be Multiple or a list of sources, a sweetener with no sweetener words, and fruit that might be mixed in or only a flavor. The next skills have to be exact on the training products before they are allowed to speak. A broader reading that is only mostly right will lower Q, because a wrong value still costs more than a blank.