plant new roots

plant new roots with courage, for in the soil of change, opportunities bloom.

Comment

Inspiration

The short story is that I'm a Korean/Chinese American who is bad at both languages but realized the importance of culture and now wants to speak with his grandparents in their tongue. So with the implementation of AI, I figured we could create more dynamic, adaptive, and engaging language learning content to facilitate the journey in addition to the existing static blog sites and word memory regurgitators. As for the longer version, that's definitely got more of an emotional backstory and kick to it, so I'll wait on that for now :) Currently, I'm also doing a fulltime language program this spring in South Korea while doing my master's in data science so this was also just me trying to really apply those two focuses here.

What it does

Users can hone their language skills (limited to English/Korean/Chinese for now but you can manually just add in the language codes for others) in various ways. First, the user can choose their target language of study and the level at which they want to practice. Then, they can choose to navigate to the "Practice" or "Learn" sections for just that. In the Practice section, there are options for Reading, Writing, Listening, and Speaking. For example, a user can select the Reading option, where they can then select a length of text that they want to be generated in that target language/level by Gemini, and then there will also be questions made for that script as well to be submitted/graded. Each option is unique, when we go to Writing, the user can select between three different scenarios: Daily, Professional/Academic, and Cultural conversations to practice in (conversing with a mixture of Dialogflow and Gemini). For every language skill option, the user can save their session and then navigate towards the "Analysis" section where they can select them and have Gemini analyze them. Once a conversation is saved, it can be quickly retrieved and reviewed for future reference. The saved analyses are tracked and visualized on the "Statistics" page, where the user can currently see the distribution of their work, sorted by date and different chart types. In addition the Practice section, the Learn section is where the user can discover new words, grammar (of course adjusted towards the user's selected level and language), and information about other cultures for greater exposure.

How we built it

React is just too good. Aside from that, I used Gemini for the novel content generation as well as analysis. Other APIs such as the Cloud Natural Language API were used to extract even more data (sentiment score, etc) from parts like the writing conversation to help Gemini develop an even better report. Dialogflow was implemented in the Writing and Speaking pages and since that requires more time to fully develop, I set the default fallback intent to send a message in which Gemini would pick up and then develop a response on its end if it came to that. The user profile, saved conversation, and analysis data is all stored in Mongo.

Challenges we ran into

  1. Modularizing Code: New technology aside, I didn't have such a swell time modularizing some of the code. For example, I originally only built this application to implement the writing feature, so when it came down to integrating parts like reading, writing, and speaking in the analysis section, I realized that I had to make the code more reusable and modularized for future updates, which did not go well for a while. The reason why I highlight this is because in the future, when I plan to integrate more descriptive statistics into the analysis models in the backend, I'll have to keep in mind the multiple files (frontend requests, backend routes, controllers, etc.) to update just because of a 3-line change in one schema, which for me, is much harder than learning to implement a new feature like Gemini or some API.

  2. Tracking Issues: Dealing with existing tracking issues was also a major mental brain twist, i.e., an easy one was removing an existing tracking to a .env file which I had to do to add it properly to the .gitignore. Still, one issue that had me stumped for a while was getting an error from a field in my database that didn't exist in the current version. While it ended up being a simple fix (running a single update line in a Mongo shell), I spent a lot of time going through my existing code which was fine at the time. And it was things like these that, if I didn't expect to inhibit my momentum, caused the most head scratches.

  3. Nontechnical Challenges: A nontechnical issue that I had was answering the question of what I wanted to get out of my time while I was here in South Korea. I had my reasons for wanting to develop this application beforehand, but I was a bit conflicted with my timing because who knows when I will have the privilege to travel back here again. But, as I kept developing pnr, I realized how deeply invested I was in it and then decided to go all in (hence the additions of other components like speaking, reading, and listening when it was originally just writing).

Accomplishments that we're proud of

This is really a project that defines an aspect of me and isn't something that I built just because I thought it would be purely convenient to have. It's the culmulation of my past struggles between computer science, finding my cultural identity, language learning and how I've been able to navigate myself through those to reach where I am now. Being able to translate those struggles into a positive application here for myself and others is something that truly excites me.

What we learned

This project wasn't just a solo project - it was the combined effort of everyone that I was able to connect with during this journey. I became a regular at some cafes and restaurants and even though they never saw a single line of code, they were the difference makers. The small daily conversations that I would have would really influence how I would approach this project and it turned from being some grand out-of-reach idea into a bridge that could connect me and them.

What's next for plant new roots

Each of the core practice pages needs their own updates because they are still far from being as comprehensive as I want them to be. For example, the Speaking page is awkward, having to start/stop recording sending messages. Also, I want to add more sub categories to each practice, such as pronunciation within speaking. Building on that, I want to add more statistic visualizations that aren't just counting the quantity of analyses, but showing user progress i.e. pronunciation score. Another big update would be to make the user content adaptive to the user past work they put in and then create/recommend a learning path. Furthermore, combining what I had in my previous project, Hanguru, and letting the user practice specific words would be a great addition. I can keep going on and on, especially when it comes to community related features like forums and whatnot, but I don't want to get too ahead of myself.

Built With

Share this project:

Updates