Malaysian Meal Identifier

Inspiration

Malaysian cuisine is incredibly rich and diverse, but many people struggle to identify dishes or learn how to cook them after seeing them in photos. Sometimes you see a dish at a restaurant, in a food video, or in someone’s social media post and wonder, “What is this dish and how can I cook it myself?”

This project was inspired by the idea of making Malaysian food more accessible through AI. By simply uploading a photo of a dish, users can instantly identify the meal, learn how to cook it, understand its ingredients, and estimate the cost of preparing it at home.

The goal is to combine computer vision and generative AI to bridge the gap between discovering food and learning how to cook it.

What it does

The Malaysian Meal Identifier is an AI-powered application that helps users identify Malaysian dishes and learn how to cook them using just a photo.

Users can upload a picture of a meal, and the system will analyze the image using a multimodal AI model. Once the dish is identified, the application generates detailed information about the meal, including the recipe, ingredients, cooking instructions, and nutritional information.

The application also estimates the Malaysian market price of the ingredients, allowing users to understand the approximate cost of cooking the dish at home. In addition, it provides dietary information such as whether the dish is halal-friendly or suitable for people with certain health considerations like diabetes or low blood pressure.

To further assist users, the system embeds step-by-step cooking tutorial videos directly within the application, making it easier for users to follow along and prepare the dish themselves.

If the uploaded image does not clearly contain a recognizable meal, the system will prompt the user to upload a clearer photo.

Overall, the application transforms a simple food photo into a complete cooking guide, helping users discover, understand, and recreate Malaysian cuisine.

How we built it

The application is built as a web-based AI-powered meal identification system.

Workflow 1. The user uploads a photo of a meal. 2. The image is sent to a multimodal AI model (Gemini). 3. The AI analyzes the image and attempts to identify the Malaysian dish. 4. Once identified, the AI generates: • The recipe • Ingredients list • Cooking instructions • Nutritional information • Dietary warnings (e.g., halal suitability, diabetic considerations) • Estimated ingredient prices in Malaysia • Total cooking cost 5. The system then retrieves cooking tutorial videos and embeds the first available playable video inside the application. 6. The results are displayed in structured recipe cards which users can sort by relevance, rating, difficulty, or cost.

Technologies Used • Gemini API for image understanding and recipe generation • React / Web frontend • Tailwind CSS for UI styling • YouTube video embedding for cooking tutorials • Prompt engineering to structure AI responses consistently

Challenges we ran into

One of the biggest challenges was accurate meal recognition.

Certain iconic Malaysian dishes such as Nasi Lemak or Char Kuey Teow are relatively easy for AI models to recognize because they have distinctive visual features. However, many Malaysian dishes can look very similar to each other, especially when: • Different dishes share similar ingredients • The food presentation varies across restaurants • The photo quality is unclear or poorly lit

For example, dishes like fried rice variations, noodle dishes, or mixed rice plates can be difficult for AI to confidently identify.

To address this, we implemented a validation step where the AI will prompt the user to upload a clearer photo if it cannot confidently detect a meal.

Another challenge was ensuring reliable cooking video links, since videos may become unavailable over time. To solve this, the system searches for multiple potential videos and automatically selects the first playable one.

What we learned

Through this project we explored how modern AI models can combine image understanding with contextual knowledge to generate meaningful outputs.

We learned how to: • Use multimodal AI models capable of interpreting images and generating structured responses. • Design prompts that guide the AI to produce consistent recipe outputs, including ingredients, instructions, nutritional information, and dietary considerations. • Integrate real-world contextual data, such as estimated Malaysian ingredient prices. • Build a responsive web interface that allows users to easily upload images and interact with AI-generated results.

This project also helped us better understand how AI can be used to solve everyday lifestyle problems, not just technical ones.