EsyLearn

EsyLearn is a virtual assistant that talks with you and answers your most pressing questions.

Comment

Inspiration

Given the recent development in AI and concern around education, we realized instead of fearing the rapid development in Large Language Models like chatGPT, we should find a way to incorporate it into our daily learning to enhance our learning. We came up with the idea to have a personal learning assistant that will accept and adjust to the learning needs of its user.

What it does

It communicates with users via a web interface where the users can easily speak and get responses back just like talking to another human being. Through natural language processing, it provides responses both in text and audio for better understanding.

How we built it

We built server based application named EsyLearn using multiple APIs listed below:

  1. OpenAI
  2. Flask
  3. Recorder.js
  4. SpeechRecognition
  5. Python text-to-speech (pyttsx3)

Challenges we ran into

It was hard to figure out the audio interactions between the user and the assistant on a server-based application. Most of the time, one API would not accept the format produced by another API. It was challenging to process audio to text and vice versa simultaneously without using local storage or storage of any kind. Staying up all night and trying to figure out the problem was challenging but rewarding.

Accomplishments that we're proud of

Making our application do what we wanted even if it's not meeting the full potential we imagined is really rewarding. Within 24 hours we essentially created a virtual assistant that is as smart as Jarvis in the Iron Man Movies.

What we learned

Never give up, add and commit regularly to the git repository, and bring a pillow. In addition, we learned to code in Flask through documentation, and blogs.

What's next for EsyLearn

Given more time we wished to implement a lot more features and make it more functional. First of all, we would like to fix the issue of not being able to interact continuously, using multiple requests. After that, it would be great to upload it on an actual domain, so that other people can access it and use it.

Other features that we hope to implement: (1)Having multiple sets/entry boxes where the user could input such things as their grade level, the speed they want the audio to be read, the voice they want the program to use, etc. so that the user can have a personalized learning experience. (2)Another feature that we were excited to implement was the use of Twilio. After we heard about it during the presentation, we thought it would be awesome for our project. The user could input their phone number and a topic they are interested in, and with the use of Twilio, the website would send one fact about the topic based on user preference, for example, once a day. This would require quite a bit of development, and implementation of databases to manage user log-ins and the information that has been sent. Sadly, we realized this was not feasible in the time frame of the Hackathon. (3)Another thing that we found out was feasible but we did not have time to do was using the SpeechRecognition module to implement multiple languages. So that the program would have a global impact, instead of being limited to English-speaking countries.

Built With

Share this project:

Updates