Enhancing Vocabulary Memory with Visual Associations

Link to Submission

https://drive.google.com/file/d/1YqVWFH1WX-yE3elrMpx2qaKyvHiuntyj/view?usp=share_link

Inspiration

We are in a university that travels to seven different countries. Therefore, around 70% of the students of our class are studying a new language ( primarily Spanish, Korean, and Chinese) through Duolingo and other digital tools like Youtube. The students often reach out to classmates who are native speakers to demonstrate their learning progress or try to approach a local in a rotation city and talk with them with a foreign language. In these moments, we noticed that a lot struggle with remembering some words that are needed to make the conversation clearer and more engaging. Therefore, we decided to choose the challenge of memorizing vocabulary.

What it does

The Use case includes ChatGPT which helps the student create prompts for generative A.I. The ChatGPT first understands what is the target language and the word. Then, it breaks the word down into parts that can be represented by English words with similar phonetics. Based on these insights, it writes a prompt that includes these English words and the translated word, so the generative A. I can generate a unique image that integrates all these elements. After, the student paste the prompt into Canva A.I and waiting for the image to be created!

How we built it

We first trained ChatGPT about the phonetics of a specific foreign language and how those phonetics can be aligned with the English words. Later, we gathered some foreign words and began to write prompts. We iterated this process for more than 5 times, and in each iteration, we switched some specific words making the prompt easier to understand. For the generative AI part, we tried different platforms like Scenario and Microsoft Bing and chose the one that created images that aligned the most with the prompt and created the most consistent images.

Challenges we ran into

• Come up with a topic: Initially, we focused on solving issues in Computer Science-related learning, and couldn't yield any innovative ideas. Thus, we spent 3 hours brainstorming
• Word choice when giving instructions to ChatGPT. Initially, it didn't do what we wanted it to do exactly. Therefore, we had to try out a lot of synonyms and writing styles.

Accomplishments that we're proud of

• We made good use of the Science of Learning research paper and implemented useful insights about the learning principles and techniques in real-world cases.
• We focused on using only free versions of the AI platforms, forcing ourselves to think more out of the box to come up with solutions that are not only affordable but useful.

What we learned

• We learned to work with constraints. At the beginning of our brainstorming sessions, we struggled to come up with practical ideas because we felt that free AI tools have limited capacity to solve real-world problems. However, we tried to list all the constraints and the resources we have and try to brainstorm from there. Eventually, we begin to get used to thinking without the premium versions of A.I tools and come up with the final solution.
• We also developed our writing skills, especially in the aspect of clarity and conciseness, so ChatGPT could understand what we wanted.

What's Next for Enhancing Vocabulary Memory with Visual Associations

• We want to further train ChatGPT about the unique phonetic characteristics of different languages. For example, Chinese has four types of pronunciation tones, which don't exist in English. So, we seek to input more information about the special pronunciation of foreign languages to ChatGPT.
• We also plan to add supplemental prompts that can make the user experience more customizable. For example, offering a prompt in which the user can input their interests and ask ChatGPT to focus on creating prompts that relate to their interests. In this way, the generated images can contain elements that the user is more interested in.

Built With

• canva
• chatgpt