SipSnap

SipSnap is a free, privacy-friendly AI drink creator that scans a fridge or pantry photo and turns available ingredients into a creative, 100% non-alcoholic mocktail, cooler, or smoothie recipe.

Comment

Inspiration

What it does

Inspiration

Many people have fruits, herbs, juices, and mixers sitting in their refrigerator but do not know how to combine them into something exciting.

Most drink-recipe tools also focus heavily on alcoholic cocktails. We wanted to create a family-friendly alternative designed specifically for people who prefer completely non-alcoholic beverages.

This inspired SipSnap: a private, accessible, and zero-proof drink assistant that turns everyday fridge ingredients into premium café-style drinks—without requiring a paid API, account, or subscription.

What It Does

SipSnap transforms a fridge or pantry photo into a personalized non-alcoholic drink recipe.

The user can:

Upload or capture a refrigerator photo. Let the on-device vision model suggest visible ingredients. Review, remove, or manually add ingredients. Generate a structured zero-proof recipe. View exact quantities, flavor profile, preparation steps, and garnish ideas. Download the recipe as structured JSON or print the recipe card.

SipSnap includes a strict Zero-Alcohol Guardrail.

Alcoholic products are excluded from the vision model’s ingredient list. If a user manually enters an alcoholic ingredient, the app blocks it before recipe generation.

Every final recipe includes:

"zero_alcohol_verified": true How We Built It

SipSnap was built as a modern, responsive web application using:

Next.js React TypeScript Tailwind CSS Transformers.js Open-source CLIP vision model Lucide React icons Cloudflare-compatible deployment

Instead of using a paid vision API, SipSnap runs ingredient recognition directly inside the user’s browser with:

Xenova/clip-vit-base-patch32

The uploaded photo is compared against a carefully selected list of fruits, herbs, spices, juices, plant milks, dairy ingredients, and non-alcoholic mixers.

The user then reviews the detected ingredients before the local recipe engine creates a structured recipe object.

The recipe engine determines:

Primary fruit Supporting fruit or herb Spice Sweetener Liquid base Drink style Ingredient quantities Preparation method Garnish recommendation

No API key, paid inference service, or server-side image processing is required.

Privacy-First Architecture

SipSnap was designed with user privacy in mind.

Photos are processed inside the browser. Images are not uploaded to our server. No account is required. No personal information is collected. No paid API credentials are needed. The downloaded vision model can be cached by the browser.

This makes SipSnap suitable for families, students, health-conscious users, and privacy-conscious users.

Zero-Alcohol Guardrail

SipSnap’s safety system works at multiple levels:

  1. Detection Guardrail

Alcoholic labels are never included in the vision model’s candidate ingredient list.

  1. Manual Entry Validation

Alcohol-related terms such as beer, wine, vodka, rum, gin, whiskey, tequila, and liqueur are blocked.

  1. Final Ingredient Sanitization

The ingredient list is checked again immediately before recipe generation.

  1. Verified Output

Every recipe returns:

{ "zero_alcohol_verified": true }

SipSnap is sober by design—not simply because of a prompt.

Challenges We Ran Into Browser-Based AI Performance

Running a vision model inside the browser avoids API costs, but the model must be downloaded on the first scan. We solved this with a clear loading state and an instant demo mode.

Ingredient Recognition Accuracy

A single fridge image can contain partially hidden items, packaging, shadows, and multiple similar ingredients. We designed the vision result as a suggestion rather than an unquestionable answer.

Users can review, remove, and add ingredients before generating a recipe.

Preventing False Alcohol Matches

Simple text matching initially created a problem because a word such as “ginger” contains the letters “gin.”

We fixed this by using word-boundary validation, allowing ginger while still correctly blocking gin.

Creating Reliable Structured Recipes

We needed every recipe to have predictable fields. We created a strict TypeScript recipe structure containing the drink name, flavor profile, description, ingredients, instructions, garnish, and verification status.

Accomplishments We’re Proud Of Built a working browser-based vision experience without a paid API. Created a polished and responsive consumer-friendly interface. Added multiple layers of zero-alcohol protection. Prevented alcoholic ingredients from entering the recipe engine. Created structured and downloadable recipe JSON. Added manual ingredient correction for better reliability. Built a complete demo flow that works without uploading a photo. Kept the user’s photo private and on-device. Created a print-friendly premium recipe card. What We Learned Local AI Can Create Useful Consumer Experiences

Not every AI application requires an expensive cloud API. Browser-based models can provide meaningful functionality while reducing cost and improving privacy.

Human Review Improves AI Reliability

The ingredient-review step makes the app more dependable. Instead of hiding uncertainty, SipSnap lets the user correct the AI before generating the final result.

Guardrails Work Best in Layers

A single instruction is not enough for safety-critical constraints. Combining detection restrictions, manual-entry validation, final sanitization, and structured verification creates a much stronger system.

Structured Data Makes AI Features More Practical

A predictable recipe structure makes it easier to display, download, print, validate, and eventually save recipes.

What’s Next for SipSnap Better Ingredient Detection

Test smaller and faster open-source vision models designed specifically for food and grocery recognition.

Dietary Profiles

Allow users to select preferences such as:

Vegan Dairy-free Sugar-free High-protein Keto-friendly Caffeine-free Allergy-aware Multiple Recipe Suggestions

Generate three different options from the same ingredient list:

Quick cooler Premium mocktail Healthy smoothie Recipe Saving

Let users save favorite drinks locally on their device.

Shareable Recipe Cards

Generate downloadable Pinterest, Instagram, and mobile recipe cards.

Nutrition Estimates

Provide optional approximate nutrition information with an appropriate accuracy disclaimer.

Seasonal Modes

Add recipe modes for:

Summer hydration Back-to-school drinks Holiday mocktails Breakfast smoothies After-school refreshments Party drink stations

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for SipSnap

Built With

Share this project:

Updates