Droidlingo

Our landing page

Inspiration

The Empire Strikes Back features the beloved character of R2-D2, a droid that speaks a language unintelligible to human characters but perfectly understood by other droids. Inspired by this concept, our group created Droidlingo, a language learning app that allows users to learn and translate into various fictional languages, including Galactic Basic, from the Star Wars universe. Our goal was to create an immersive language learning experience that was both fun and engaging for fans of the franchise, while also using cutting-edge technology such as deep learning and augmented reality to provide a unique translating experience. In this way, our app bridges the gap between science fiction and real-world language learning, just as R2-D2 bridged the gap between droids and humans in the Star Wars universe.

What it does

Our web-based project can be thought of as a "one-stop shop" for all our Galaxy's linguistic needs. Our homepage allows the user to do one of 3 things – translate through text, translate in real-time, or learn a language. Our text translation tool gives the user the ability to type up their thoughts in English and translate them to Galactic Basic so they can easily text their intergalactic friends! The next feature, real-time translation, uses OCR to perform real-tme translation right through their device's camera. Gone are the days of having to guess what the signs read in Hoth! Lastly, our language learning feature allows the user to learn one of many new languages by learning to translate common sayings!

How we built it

We used NodeJS for our backend with the fastify with frameowork. Our frontend was built using JavaScript and HTML and our OCR text detection was built using Python opencv, trained using pytesseract and hosted on a Flask server. We used Glitch.com to host our site.

Challenges we ran into

In our implementation images using JavaScript can only be taken and sent via the SSL protocol so we needed to install and self sign a web certificate so can send and receive image data back and fourth from our Flask server.

We had trouble using our machine learning model on Glitch.com since it limits us to about 512MB of ram and only 200MB of disk storage. This is why why had to create a microservice using Python to extract and send image data.