Catch of the Day (team #17)

Inspiration

When we moved to the UK we were concerned about how much food is wasted by the households and restaurants. We wanted to try to solve an issue in at least one of the sectors of the food production chain.

What it does

Our application helps both the food services companies and consumers. The food service companies can upload photographs and audio description of the meals which are left at the evening. This informations is processed and at the end there is a carousel with the classified food photos and transcribed description. On another hand, the consumer searches for a specific category and choose from what is available.

How I built it

Backend:

• Python 3.7;
• 3 instances of flask servers (bot, image model daemon, audio model daemon);
• ngrok
• pymessenger framework (https://github.com/davidchua/pymessenger)

Image NN model:

• Food-101 dataset (https://www.vision.ee.ethz.ch/datasets_extra/food-101/)
• Transfer-learning applied to AlexNet model

Audio NN model

• fairseq pretrained implementation (https://github.com/pytorch/audio/tree/master/examples/interactive_asr)

Challenges I ran into

Digging through PyTorch documentation to make model to work, deployment of everything to run locally well.

Accomplishments that I'm proud of

It works!

What I learned

• Messenger API (it's huge, and well-documented thou)
• Some basics of PyTorch (not so bad for a new comer into new framework)

What's next for Catch of the Day

There are a lot of improvements which might be applied:

• Build a CRM for food services to track sells/positions count tracking
• Provide a way to direct purchase positions
• Improve UX

Built With

• flask
• ngrok
• python
• pytorch