Zaika— AI Recipe Generator

"What's in your kitchen? We'll find what to cook."

Comment

Inspiration

Most "what can I cook with X" tools are either a keyword search over a recipe blog or a thin wrapper that asks an LLM to invent a recipe on the spot — which means made-up quantities, hallucinated steps, and no way to verify any of it. We wanted something closer to how real recommendation systems work: a trained model that actually learns which ingredients belong together, paired with a cooking assistant that answers follow-up questions ("can I make this vegan?") by retrieving from real, grounded knowledge instead of guessing.

What it does

You type in what's in your kitchen — tomato, onion, garlic, rice — and Pantry ranks recipes by how well they match, using a dual-encoder embedding model rather than plain keyword matching. Each result shows exactly what's missing, cook time, difficulty, and full nutrition. Open a recipe and there's a built-in chat assistant, grounded in a retrieval knowledge base, that answers questions like "can I replace the butter?" or "how do I make this diabetic-friendly?" with a sourced, traceable answer instead of a hallucinated one.

How we built it

Model: a weight-shared Siamese encoder trained with in-batch contrastive (InfoNCE) loss — ingredients and recipes are the same modality (an unordered ingredient set), so one shared encoder maps both into a common embedding space. Training pairs are self-supervised: a recipe's own ingredient list, randomly subsampled, stands in for "a user who has some of these ingredients," with the full list as the positive match. TensorFlow/Keras end to end, with a real hyperparameter sweep (embedding dim × learning rate) logged and compared, EarlyStopping/ModelCheckpoint/ReduceLROnPlateau/TensorBoard wired in, and evaluation via precision@k, recall@k, and MRR rather than text-generation metrics like BLEU that wouldn't fit a ranking task.

Retrieval: FAISS (IndexFlatIP) over the trained embeddings for recommendation, plus a second, separate TF-IDF retrieval index for the RAG assistant — a curated substitution knowledge base and auto-generated per-recipe summaries, with extractive (not generative) answer synthesis so every answer is traceable to a real source.

Backend: FastAPI, cleanly layered — ml/ (training-time code) never imports FastAPI, and app/ (routes → services → inference) never contains training code. Model and index load once at startup, not per request.

Frontend: React + Vite + Tailwind, with a deliberately non-default visual identity — a "cast iron" dark theme, mustard/brick/sage accents instead of the usual cream-and-terracotta AI-app palette, and a punch-hole recipe-index-card as the one signature design element on the detail view. Dark/light mode both properly theme-aware, favorites persisted locally, and a chat widget wired straight into the RAG backend.

Challenges we ran into

  • Keras 3 + legacy H5 serialization fought us hard. Automatic mask propagation (mask_zero=True) inserted an op the H5 saver couldn't reliably reload, and even after replacing it with explicit masking layers, raw Python lambdas inside Lambda layers failed to deserialize their closures correctly. Fixed by writing masking as ordinary layers and registering named functions with @register_keras_serializable instead of inline lambdas.
  • No access to Kaggle from the build environment. The real Food.com dataset (~180K recipes) lives on Kaggle, which wasn't reachable, so we built a schema-accurate synthetic sample generator instead — same columns, same value formats — so every downstream script (cleaning, vocab, training, evaluation) was built and tested against something real-shaped, and works unmodified once the real CSVs are swapped in.
  • A real lexical bug in the RAG assistant: asking to "substitute" butter wasn't matching a knowledge-base entry titled "butter substitution" — TF-IDF has no stemming, so those are different tokens entirely. Diagnosed by testing retrieval directly, fixed by broadening keyword coverage rather than adding a stemmer for one bug.
  • A theming bug that looked fine until it wasn't: dark mode worked, but light mode initially rendered dark card text on a dark card background, because only the page body's color flipped, not the component-level utility classes. Fixed by rebuilding the whole palette on CSS custom properties that flip together, instead of static per-component colors.

Accomplishments that we're proud of

Every number in this project is real, not aspirational — the model was actually trained and evaluated (67% recall@1, 93% recall@10 on the held-out test split), the API was hit through a real test client against the real trained model (22 passing tests, zero mocks), and the frontend was actually screenshotted in both themes to catch bugs before calling it done. We'd rather ship something smaller that's fully verified than something bigger that's aspirational.

What we learned

That "production-quality" mostly shows up in boring places: a masking implementation that survives serialization, a response schema that doesn't silently omit a field, a CORS policy that isn't just "*". Also that RAG's failure modes are often not about the retrieval algorithm at all — our worst bug was a stemming mismatch, not a ranking problem.

What's next for Pantry — AI Recipe Generator

Retrain on the full ~180K-recipe Food.com dataset now that the pipeline is proven end-to-end; add hard-negative mining once random in-batch negatives get too easy at scale; swap TF-IDF for a neural retriever in the RAG assistant once paraphrase robustness matters more than the added dependency; and layer in real collaborative filtering once there's enough interaction data to justify it alongside the content-based retrieval that's there today.

Built With

Share this project:

Updates

Submission history