Makeable - Hackathon Submission

Inspiration

Cooking at home sounds simple.

Until you actually try to do it.

First, you spend 20 minutes figuring out what to eat. Then you check what’s in your fridge. Then you try to remember which recipe you saved on Instagram, or was it TikTok? Or maybe it was in that PDF cookbook you downloaded months ago. Or a random website with 15 popups.

There’s no single place where everything lives. It’s exhausting.

The most frustrating part? When you’re actually cooking.

Your hands are messy. The phone isn’t nearby. You keep going back and forth between ingredients, steps, preparation notes, and timers. You just want to know: “How many potatoes was that again?” — without touching anything.

We wanted cooking to feel less lonely.

And thus that is why Makeable was born.

Makeable isn’t just a recipe app. It’s a cooking partner that stays with you from discovery to dinner — and makes the whole process feel effortless instead of overwhelming.

What it does

Makeable is your AI cooking companion — built to handle everything from recipe discovery to the moment you finish eating.

AI Cooking Companion

Cooking gets chaotic. Your hands are messy. Your phone isn’t nearby. The screen keeps locking. You keep scrolling back to check the next step.

With Makeable, you don’t have to.

Just ask.

“How many potatoes?” “What’s the next step?” “Set a timer for 8 minutes.” “Do I already have soy sauce?”

It answers instantly — no unlocking your phone, no jumping between screens. It feels like someone is standing next to you in the kitchen, helping you.

Universal Recipe Import

Recipes live everywhere — TikTok, Instagram, YouTube, random food blogs, PDFs, handwritten notebooks.

Makeable brings them all into one place.

Paste a link from TikTok, Instagram, YouTube, or any website
Use our in-app browser to discover recipes and save them seamlessly
Scan a photo of your grandma’s handwritten recipe

Everything becomes structured, searchable, and ready to cook.

The Complete Cooking Lifecycle

Makeable doesn’t stop at saving recipes. It connects the entire cooking journey.

Choose recipes: Pick what you want to cook this week.
Add to your meal plan: Drag them into your weekly calendar.
Automatic grocery list generation

We combine ingredients across recipes and generate a smart grocery list.

If something is already in your pantry, we don’t add it.

If you only have half, we calculate the rest.

If something is expiring soon, we tell you.

One-click pantry tracking after shopping

Bought your groceries? Scan barcodes and add items instantly.

Our proprietary software even estimates expiry dates based on product type — something most apps don’t even attempt.

Cook and track

Mark meals as cooked.

Track leftovers.

See what you’ve made.

Reduce waste.

How we built it

Frontend

React Native with Expo for cross-platform iOS/Android development
TypeScript for type safety throughout the codebase
Zustand for efficient state management with optimistic updates
Livekit with WebRTC for streaming voice for ultra low latency
Zod, because I love zod

Backend Infrastructure

Firebase Admin SDK for backend authentication and database access
Firestore for real-time database with offline support
Firebase Storage for recipe images and cookbook uploads
Custom caching layer (global and user-specific) for performance optimization

Voice Infrastructure

Express.js backend server for long lived sessions
LiveKit Agent Server for managing rooms and participants
Deepgram STT (Speech-to-Text) with Nova-2 model for transcription
Deepgram TTS (Text-to-Speech) with Aura voices (Luna/Arcas) for natural speech
Silero VAD (Voice Activity Detection) for detecting when users speak
Azure OpenAI GPT-4o for conversational AI and recipe assistance
Dual agent modes: Recipe-specific and General assistant

AI Integration

OpenAI GPT-4 for recipe extraction from URLs and text
GPT-4 Vision for scanning physical cookbook pages
Azure OpenAI GPT-4o for voice conversations and real-time assistance

Additional Technologies

RevenueCat for subscription management and paywall
React Native Vision Camera for barcode scanning
OpenFoodFacts API for barcode product lookup (opensource)

Challenges we ran into

1. AI Recipe Extraction Accuracy

Recipe videos are chaos.

There is no standard format. No guaranteed structure. Sometimes the ingredients are in the caption. Sometimes they’re buried in the comments. Sometimes they’re spoken in the video but never written anywhere. And sometimes they’re incomplete altogether.

Even when we gathered everything, it was messy and inconsistent. Quantities were vague. Steps were implied. Ingredients were referenced casually (“add some butter”) without exact measurements.

If that wasn’t hard enough — we also needed it to feel fast.

AI inference can easily take 10–20 seconds if you’re not careful. That’s unacceptable for a mobile app UX. So we broke the extraction process into smaller structured passes and ran multiple AI sessions in parallel, each responsible for a specific part (ingredients, steps, metadata). Then we validated and merged the outputs.

2. Meal ↔ Pantry ↔ Grocery Integration

Connecting meal planning, pantry tracking, and grocery generation sounds simple.

It is not.

There are thousands of edge cases when you build an end-to-end lifecycle:

What if an ingredient exists in the pantry but in a different quantity?
What if the pantry has “2 tomatoes” and the recipe needs 300g?
What if a barcode scan returns a product we’ve never seen before?
What if two recipes reference the same ingredient differently (“red onion” vs “onion”)?
What if an item was marked purchased but rejected to be added to pantry?
What if a user edits a recipe after the grocery list was generated?

Every single one of these scenarios creates cascading state changes across systems.

Synchronizing Meal Plan ↔ Grocery ↔ Pantry in real time, with optimistic UI, without corrupting quantities — required a centralized state model and extensive defensive validation.

This part alone generated hundreds of bugs during development.

3. Data Quality Problems

Food data is wildly unstructured.

There is no universal convention for how ingredients are written.

Is it:

“1 cup chopped onions”
“Onions, 1 cup (chopped)”
“1 onion”
“200g onion”
“Half a large onion”

They all mean slightly different things.

Matching ingredients from videos, Photos, Websites, Handwritten recipes to ingredients already in our database was extremely challenging.

Getting this wrong breaks the entire lifecycle.

Food Shelf Life

There is no simple, reliable public dataset that tells you exactly how long every food item lasts in every storage condition.

We had to:

Integrate multiple open-source datasets
Add our own manual curation
Build heuristics for estimating expiry (especially for barcode-scanned products)

Even then, edge cases constantly appear.

Units & Conversions

Cooking units are a nightmare.

Grams, ounces, cups, tablespoons, pieces, cloves, bunches — and they’re not interchangeable without context.

A “cup of spinach” is not the same weight as a “cup of flour.”

When you’re building an end-to-end system (meal → grocery → pantry → leftovers), unit inconsistencies create subtle bugs everywhere.

Hundreds of issues surfaced just because of unit conversions and rounding logic.

4. Building Our First Mobile App

This was both of our first serious mobile apps.

Mobile development is very different from web.

Even though tools like Claude helped accelerate development, but it still came with its own complexity, especially when:

Configuring native builds
Deploying to TestFlight
Managing Android signing and Play Console requirements

But overcoming these challenges is what makes the system feel polished today.

Accomplishments that we're proud of

1. Post-Import Smart Workflow After importing a recipe instead of just saving it, we immediately help users:

Check if they have ingredients in their pantry
Add missing items to grocery list
Schedule the meal for this week (if chosen)
Start cooking right away This single feature transforms recipe discovery into action.

2. Voice-First Cooking Experience Our hands-free voice interface during cooking is magical. Users can say "next step," "set a timer for 10 minutes," "how much garlic?" without touching their phone. The AI understands context and responds naturally, making cooking feel like having a sous chef.

3. Barcode Scanning Integration Our pantry barcode scanner with OpenFoodFacts integration makes inventory management effortless. Scan a product, and we automatically populate name, image, and typical expiry dates.