FlashMonsters - Bring new words to life.

• 

  Slide Deck Logo

• 

  GenAI Pipeline

• 

  Sample User Flow

Inspiration

Are you still struggling to incorporate new words daily when learning a language? You try to create flashcards to test yourself, but only reading the same word from a paper repeatedly doesn't help you associate meaning with those written characters. But what if you combine text with images and sound to bring this word to life? With FlashMonsters, you can fight back your fears of practicing vocabulary building by collecting creative, AI-generated monster friends, which help you associate a word with an image and sound, going far beyond simple text recognition.

Inspired by the science of learning, FlashMonsters uses multimedia learning to enhance memory retention. The brain stores multiple representations in memory (some verbal, some visual—which are stored in different parts of the brain), which gives you multiple shots at later digging the information out of storage*. On the other hand, the gamified components were inspired by PokemonGo, Quizlet, and the character design by Monsters Inc., given its creative appeal to young users. Our target audience is K-12 students, yet the game welcomes all ages alike!

*Kosslyn, S. M. (2017, October 6). The Science of Learning: Mechanisms and principles. MIT Press. https://direct.mit.edu/books/edited-volume/4481/chapter-abstract/191666/The-Science-of-Learning-Mechanisms-and-Principles?redirectedFrom=fulltext

What it does

A sample use flow for the FlashMonsters MVP has been included in the Project Media section below.

How we built it

We started our development process by selecting which features FlashMonsters would include. Beyond discussing amongst ourselves, we gathered insights about language learning experience from Marina Mogilko (LinguaMarina and Silicon Valley Girl) and Anthony Giuliani (TwelveLabs). From that step, we refined our Generative AI pipeline to a sequence of steps defined in the Project Media files below.

In sequence, we started searching and testing AI models across the different modalities we sought to include in the app. We then continued with our development process on FlashMonsters using the following combination of tools:

• Back-End: FastAPI, Python, Firebase;
• Design: Figma, inspired by flashcard apps such as Quizlet and PokemonGo;
• Mobile Development: FlutterFlow;
• AI models:
  • GPT-4;
  • Stable Diffusion XL;
  • MusicGen;
  • ElevenLabs.

Challenges we ran into

1. Brainstorming how to incorporate multimodal AI tools into a vocabulary-building application in a way that would be meaningful and effective for the user's learning experience. We tackled this after seeking external input from Marina and Anthony, as mentioned above, during the hackathon;
2. Adapting UI design decisions given the limitations of FlutterFlow, which we solved by simplifying the dynamic elements of the UI, such as buttons and scrollable pages, but including additional decorative details, such as gradients and color overlays over the cards, which enhance the overall visual outlook of the application;
3. Running compute-heavy AI models on the cloud, which we solved using Replicate;
4. Coping with bugs on FlutterFlow, given that it's still a new, no-code tool.

Accomplishments that we're proud of

Considering we are all international first-year college students being introduced to this new and fascinating world of hackathons, we are genuinely proud of delivering this entire project to you today. Yet if we were to point out a particular aspect, we would say we're especially proud of combining different Generative AI modalities to improve a user's language learning experience.

What we learned

1. How to develop a project in 24 hours that incorporates the latest cutting-edge AI technology being developed here, at the heart of the Bay Area and Silicon Valley;
2. How sincerely incredible it is to be a part of a supportive, energizing community of ambitious innovators willing to leverage advanced technologies to leave the best possible impact on the world!!

What's next for FlashMonsters

1. Improve the Spaced Repetition algorithm to automatically schedule battles for the user;
2. Include speech-to-text recognition to check for a user's pronunciation (adding a new type of question on the "battle" feature).
3. Adding features to enable the sharing of vocabulary cards among users, including the ability to battle other people's monsters to encourage collaboration and friendship-building outside the screen.

Built With

• elevenlabs
• fastapi
• flutterflow
• gpt-4
• musicgen
• sdxl