Kids nowadays are very familiar with smart phones, but this knowledge is not exploited in the right way, because they're mostly consuming non enriching content. Kids are also naturally fast learners especially when it comes to new languages, they are also known to have a very strong visual memory. Hence the idea "Learn With Ako!" was born.
What it does
"Learn With Ako!" give the child the ability to explore his environment while taking pics, it recognizes the objects shown in the picture and generate a caption about it, helping him describe his environement in the new language he is learning. "Learn With Ako!" doesnt only do this, it also tests the child knowledge through quizzes, where it shows him a picture and ask him to fill in the blanks to complete the picture's caption.
How we built it
we used 3 datasets: "COCO MS" to implement the objet detection model. "Flickr 8k" to implement the caption generator model. "The Europarl dataset" to implement the translator. after generating the three models we joined them so they can detect the objects in the picture and
Challenges we ran into
we didn't really know what datasets are, but we could figure out how we can exploit them in training a model that recognizes objects from a scenery, and create meaningful sentences.
Accomplishments that we're proud of
a very attractive and intuitive UI/UX designed with Figma and later built with Flutter. a model that recognizes objects from a picture with Python.
What we learned
we learnt how to use datasets to solve real life problem.