ReadEDU

user login page
main menu page
story reading
user information page

Inspiration

As children struggles at school to read alongside their peers and do not have much support in learning. We want to apply current LLM AI technology to assist and boost their language learning.

What it does

It could recognize the users' speech regarding the story context, auto check and evaluate the mistakes the user made, and adjust the story difficulty level based on the user history performance.

How we built it

We applied general human-computer interaction design principles to design our app:

Created storyboards as low-fidelity prototypes to visualize our design ideas.
Conducted user surveys based on the storyboard interactions to test the functionality, discovering the potential flaws of the initial design.
Tested our LLM AI model with a testing dataset and fine-tuned the parameters for our design scenario.
Used Figma to create a mid/high-fidelity prototype, visualizing our frontend UI interfaces.
Implemented the UI with frontend tools and Model API, building the database structure.

Challenges we ran into

Since the target users are young kids, they have difficulty understanding instructions during the user survey, making it hard to receive clear feedback.
We originally wanted to apply OpenAI Whisper as the AI model API; however, it is a relatively complex model, and we have limited resources to achieve our desired performance. The model has many parameters, and even testing it across a small dataset requires a huge amount of time. Our computers are all MacOS systems with ARM architecture, which does not have GPU, thus limiting the computational power. Even after switching to the Spark model, which is a better fit for our case, there is always a latency in transcription, which does not meet our desire for live transcription and correctness checks.
Limited access to resources. Model API requires payment, and many tools we used have access limitations.
Difficulty linking the database to the API server due to version incompatibility of MySQL and the frontend environment.

Accomplishments that we're proud of

Deployed an applicable beta version with a well-organized UI.
Constructed the framework for future development. Users can register themselves and add stories to the database. Developers can switch the API they want to use just by linking the API key to our terminal.
Received positive feedback from the experiment participants.
Target users can easily use our app.

What we learned

Product design and project management strategies.
Technical skills like frontend UI design, API building, and machine learning model testing.

What's next for ReadEDU

Further develop the UI functionality and apply the machine learning model to adjust the difficulty level based on user performance.
Apply LLM model to auto-generate stories into the database.
Fine-tune the AI model for our case to achieve real-time transcription and correctness checks, achieving lower hardware requirements and being able to apply to various cases.