We were inspired by the creativity of the developers of Wordle, as they created a game that became a worldwide sensation. We also took inspiration of the rise of emerging technologies such as Deep Learning, AI, and Computer Vision, which served as the basis for our hackathon project.
We also saw the problem of many unfortunate refugees and foreign citizens not knowing what to call certain objects in English. We decided to solve this problem by combining the power of Deep Learning and a fun game.
We are aware of other applications like Duolingo that try to teach people different languages, however they do not teach picture to word association and word spelling which are integral to learning a new language. Additionally, .jpegle gamifies the learning experience providing for a more engaging lesson. The combination of Deep Learning and Web Development allow users to learn about any picture, in any setting, anywhere in the world.
What it does
Our project utilizes both Deep Learning and Computer Vision to generate Wordles, specifically through the popular object detection algorithm YoloV3 using a Darknet CNN for Object Detection that was utilized in our project. The object detection model is first run on the user uploaded image and an object like a "car" is randomly picked from all these detected objects. This randomized object becomes the Wordle that a player needs to guess in 6 tries. The purpose of this application is to incorporate visual learning methods to create an engaging educational game for English learning immigrants, as visual learning and gamification of learning tasks have been proven to be the most enriching forms of learning.
How we built it
We built it utilizing Keras, MongoDB, Flask, and various web development languages such as HTML, CSS. We first replicated the Darknet CNN model in Keras and loaded on pretrained weights. The CNN was then fine tuned on the MS COCO dataset and deployed. We utilized MongoDB to store the images uploaded by the user and retrieved image data from the MongoDB database for object detection. After the YoloV3 was able to perform object detection, we are able to randomly pick an object to play wordle with. Contrary to the original Wordle, our Wordle is able to support more than just 5 letter words, which allows for a better learning experience for our user, and overall a better way to achieve our goals.
Challenges we ran into
Some of the challenges that we ran into include how there were issues with the implementation of Wordle onto a Website. Specifically, we initially used pygame as a way to create our Wordle application, but as our project grew in magnitude, we needed to use a website for us to have a more impactful project.
Accomplishments that we're proud of
Some of the accomplishments that we are proud of include how we utilized various emerging technologies in the span of a single day to create a unique object detection based Wordle, something that hasn't been attempted by anyone. We are also proud of how we were able to create a project that was able to potentially serve as a benefit to the many English language learners in our society, as visual learning has proven to the most effective way of learning a new language.
What we learned
We learned about the usage of Flask and MongoDB in combination in each other, as they were useful for the development of our project. We also learned about the various integrations of web development and machine learning.
What's next for Object Detection Based Wordle Using ML and Computer Vision
We will continue to develop our idea to be expanded to an even greater audience.
Log in or sign up for Devpost to join the conversation.