Memody

Inspiration

The inspiration for Memody came from understanding how the human brain processes information through multiple sensory channels. Research shows that multi-modal learning - combining visual and auditory inputs - can significantly enhance memory retention and comprehension. This will be extraordinarily helpful for learners with dyslexia and ADHD. We were also inspired by how music has been used throughout history as a memorization tool, from ancient oral traditions to modern educational songs. By transforming text into music, we're creating a new way to experience written content that leverages these powerful cognitive connections.

What it does

Memody transforms the reading experience by creating a synchronized audio-visual experience:

Real-time text highlighting follows your natural reading pace, using a karaoke-style visualization
Converts text into melodic patterns based on sentence structure, word importance, and semantic meaning
Creates unique musical signatures for different types of content (articles, documentation, educational material)
Offers customizable audio settings to match user preferences and content type
Works seamlessly across any webpage through the browser extension

How we built it

JavaScript-based browser extension framework for cross-browser compatibility
Text analysis engine to process sentence structure and time words to music(music.ai)
LLM to generate music corresponding to text
Interpretative layer LLM for simplifying dense text

Challenges we ran into

Sourcing an LLM to generate music corresponding to text
Deciding on the optimal purpose of the application (enhancing learning and memorization with multi-modal learning vs. emphasis on interpreting the text into catchy tunes)
challenges of matching natural sounding music to reading speed

Accomplishments that we're proud of

Created a novel approach to enhancing reading comprehension through music
Developed a text-to-music system that produces coherent, pleasant melodies
Built a smooth, intuitive user interface that works across various websites without significantly impacting user performance

What we learned

the importance of the user experience in educational tools
adjusting between performance and functionality, making tradeoffs
The challenges of sourcing technology for cheap in a burgeoning field

What's next for Memody

Real-time text tracking system for accurate highlighting
- Enhance the tool specifically for people with ADHD and dyslexia
- A custom algorithm that further aligns text patterns with music
- Build analytics to track reading comprehension improvements
- Add support for different languages