Food Scanner

Architecture-Diagram
FoodScannerCICD
sample-response

Inspiration

To empower people with knowledge about what they eat and help combat the rising consumption of unhealthy, processed foods.

What it does

Recognises any type of food, gives food composition, ingredients, summary, nutritional information, and healthiness assessment.

How we built it

First, we identified local foods that the foundation model performs poorly in recognizing, then we trained a custom efficient net model, to classify these foods, then added the LLM ( Claude Sonnet 3.7), to give all the relevant information about the food. It begins to perform well once we removed the bottleneck of having to recognise the food.

Challenges we ran into

Initially we wanted to use the Claude Sonnet 3.7 model directly, but when we found out about the underperformance on local foods, we had to find a way to solve this, which we did with the custom computer vision model.

Accomplishments that we're proud of

Being able to help local businesses, creating health awareness around food consumption and solving real world problems that affect human lives!

What we learned

We learnt that Amazon Bedrock model's performances can be enhanced further by combining them with other models (in this case our custom model acting as a RAG application). We also learned about the power of AWS SAM in simplifying the building and deployment of serverless applications on AWS!