Recipe-to-Reality

Inspiration

The idea for this project arose from the hassle of managing ingredients while trying out new recipes from YouTube. Juggling between watching the video, jotting down ingredients, checking what I already had at home, and adding items one by one to cart was tedious and prone to mistakes. This got me thinking: Could technology simplify this process? Better yet, Could AI make it seamless? This question inspired us to create an AI-powered tool designed to streamline and enhance the entire experience.

What it does

Recipe-to-Reality is a Chrome extension that:

1. Extracts recipe details from YouTube video descriptions.
2. Creates an editable shopping list, letting users remove items they already have.
3. Suggests alternative ingredients using AI
4. Prepares a shopping cart on Blinkit (a quick commerce website widely used in India) with available ingredients.
5. Highlights items that couldn’t be added to the cart for transparency.

How we built it

1. Creating the Shopping List:
   • Utilized the built-in AI "PROMPT API" to extract a structured shopping list from the YouTube video description of the currently open page.
   • The extraction is triggered when the user clicks the "Want to order ingredients?" button, which initiates a message from the content script to the background script for AI processing.
   • Leveraged a MutationObserver to detect when YouTube video descriptions change dynamically, ensuring the extension remains responsive to user navigation.
2. Suggesting Alternatives for Ingredients:
   • Used the built-in AI "PROMPT API" to suggest alternatives for ingredients based on user preferences, dietary restrictions, or availability.
   • Alternatives are displayed alongside the shopping list, allowing users to choose substitutes if desired.
3. Adding Ingredients to the Cart:
   • Automated the process of adding ingredients to a Blinkit shopping cart using a multi-step approach:
     • Search for Ingredient: The extension navigates to Blinkit's search page and programmatically types the ingredient into the search bar.
     • Shortlist Best Product: The built-in AI "PROMPT API" evaluates the top three product results from Blinkit’s search and selects the most relevant item based on its name and description.
     • Add to Cart: The extension identifies and clicks the "ADD" button for the selected product, ensuring it gets added to the user’s cart.
   • Error handling ensures unavailable items are logged and displayed to the user for transparency.
4. Frontend:
   • Designed a simple and intuitive UI using HTML, CSS, and JavaScript.
5. Chrome Extension Framework:
   • Developed on the foundation of Chrome’s extension APIs using Manifest V3 for better security, performance, and up-to-date standards.
   • Leveraged Chrome’s background.js for persistent messaging and content.js for DOM interaction on YouTube and Blinkit.
6. Communication:
   • Used message passing between background scripts and content scripts for seamless integration:
     • Content Script: Extracts YouTube video descriptions and interacts with Blinkit’s DOM
     • Background Script: Manages AI queries and error handling.
7. Integrated fallback mechanisms to handle missing or unclear AI responses.

By blending advanced AI capabilities, web automation techniques, and intuitive design, we built Recipe-to-Reality to simplify the cooking experience for users. This fusion of technology ensures a seamless and hassle-free transition from recipe exploration to ingredient acquisition.

Challenges we ran into

1. Unstructured Data:
   • YouTube video descriptions are often unstructured and inconsistent, with ingredients buried in a mix of text, links, and promotional content. Extracting meaningful information required leveraging AI for natural language processing and implementing robust parsing logic to handle edge cases.
2. API Limitations:
   • Many shopping platforms, including Blinkit, lack public APIs for programmatic access, necessitating direct DOM manipulation to perform tasks like searching for products and adding them to the cart.
   • This approach, while effective, is less efficient and more error-prone compared to API integration. Access to public APIs could significantly enhance the extension's performance and reliability.
3. DOM Fluctuations:
   • Blinkit’s frequently updated DOM structure posed challenges for maintaining consistent selectors and automating interactions.
   • Regular updates to our codebase were needed to adapt to these changes, ensuring the extension remained functional over time.
4. Timeouts and Latency:
   • Managing asynchronous operations while scraping data, searching for items, and interacting with the cart required precise handling of timeouts and delays.
   • Ensuring a smooth user experience meant balancing speed with reliability, particularly in scenarios with slower network connections or delayed responses from the server.
5. AI Prompt Errors:
   • Occasionally, the built-in "Prompt API" threw errors such as:
     • Error processing prompt: NotSupportedError: The model attempted to output text in an untested language, and was prevented from doing so.
   • To address this, we implemented exception handling that triggered a fallback to a traditional non-AI approach (e.g., keyword matching and predefined rules) when such errors occurred.
   • While these errors were rare, they highlighted the importance of resilience in AI-driven applications.

Despite these challenges, each obstacle helped refine our approach and contributed to the robustness and adaptability of the extension

Accomplishments that we're proud of

• Integrated the Built-in "Prompt API" for seamless ingredient extraction and alternative suggestions and finalizing product
• Automated the ingredient search and cart addition on Blinkit, using AI to select the best products.
• Developed a lightweight and intuitive UI with HTML, CSS, and JavaScript for smooth interaction with YouTube and Blinkit.
• Incorporated exception handling for AI errors, ensuring functionality by reverting to a non-AI approach when needed.

What we learned

We learned how to effectively integrate AI for ingredient extraction and cart automation, navigate unstructured data, and deal with asynchronous operations and DOM manipulation. The experience also highlighted the importance of building flexible systems that can adapt to frequent changes in external platforms.

What's next for Recipe-to-Reality

Moving forward, we aim to improve speed and reliability by integrating public APIs from quick commerce platforms like Blinkit. This would streamline the process, reduce errors, and enhance the user experience. We also plan to make the extension more user-friendly with a more refined UI, better error handling, and quicker processing times, ensuring a seamless experience across various platforms and devices.

Built With

• ai
• css
• dom
• html
• javascript
• prompt-api