Memes have become the symbol of trend. Trends are the best thing one can have on social media and a game of trends can definitely serve the taste of the social media users. Popular games like Among Us, Scribble got immense popularity due to the simplicity and trendy activities. All these thoughts inspired us to work on a game based on memes and how far people can connect themselves with memes.
What it does
The web application asks the user to click a snap of their faces when they are giving the best facial expression immitating the facial expression in the randomly generated meme and image is fed into an ML model which checks the similarity between two pictures and generates the score. There are four rounds and the summation of the scores is displayed as result.
How we built it
We built the image similarity feature by using Convolution Neural Network. We used two models via transfer learning to make the DL model with custom features. It generates the cosine distance between two images( user's and meme image). We used one pretrained and other deepnet model for this. We referred to the research paper 'A Compact embedding for facial similarity' from Google AI researcher Raviteja Vemulapalli (firstname.lastname@example.org)
Aseem Agarwala (email@example.com)
Dependencies: pytorch keras scikit-learn Flask Webcam.JS
Challenges we ran into
Initially we tried the model by using tensorflow, but the model was not deployed well and it did not produce a good output. We were facing problems while implementing back end and front end, since there were issues while capturing webcam snap. .
Accomplishments that we're proud of
We referred a research paper to implement the entire model. We used the findings from the research paper to frame the model and built the web application to access the services of the model using the MVC design pattern
What we learned
Explored modules like keras, pytorch, and other dependecies like Webcam.JS for transfer of media from front-end to back-end Sometimes it becomes difficult to put the ideas into projects just by watching tutorials. Therefore, we tried something new out of the box
What's next for MemeposterSelfieHacks
We are expecting for positive feedback on the project and also aspire to work on the valuable feedbacks and suggestions to increase the scalability of the web application and can be hosted for public use