Inspiration

Due to COVID-19 the World is facing a teacher shortage like never before. Due to pandemic most teachers won’t stick around. In person classes have been replaced with online classes and some students are finding it hard to cope up with. Both of these factors are pushing towards a future of personalised learning for the children powered by AI & Machine Learning. This prompted us to solve a problem at scale, using the knowledge we have. Everybody knows that with the age of Instagram and Twitter our attention span and desire to retain information and knowledge has become more unforgiving. Let us suppose you want to learn about photosynthesis. If given an option between reading a book and watching an animated video on it what would you choose? It is fairly obvious that a large majority of us would choose to watch a video?

What if there was a way to convert any wikipedia page to an animated video? Izzy does it takes text from any source and converts it into a video which makes learning fun.

Here is one such video: - https://wisdom-of-crowd-airtable.s3.amazonaws.com/5f9c961dfe45d8402bc89c30.mp4

What it does

Izzy is an AI powered learning management system & virtual teaching assistant that helps students learn 1-on-1. The Students and Teachers register on the platform to get started, their typing patterns are recorded and saved. The administrator of the educational institution then allots classrooms to all the students and assigns teachers to different courses & classrooms. The teachers upon logging in securely, have access to an all powerful multipurpose dashboard that shows the courses the teacher is currently teaching and a calendar. The teachers can create video assignments by following the following steps:-

  1. Give a name to the assignment
  2. Pick a deadline date
  3. Allot a classroom & a course
  4. Give a Wikipedia page name
  5. Choose 4-5 images
  6. Reorder images
  7. Click Generate Video

After clicking the teacher is directed to the Questionnaire Builder, where the teacher is presented with a set of questions that have been automatically created using advanced Natural Language Processing techniques. Not only that, the wrong options for each of these multiple choice questions are also generated by Artificial Intelligence. While the teacher reviews all the questions, the video is processed in the background, as soon as the video is created, the teacher is alerted. Cool, right? Thus saving time of the teacher that may have been spent on the tedious task of creating assignments or videos. Questions generated using advanced Natural Language Processing techniques like Neural Question Generation help in faster creation of assignments, as the teachers don't have to think of wrong answers or form questions. After editing the questions, the teacher publishes the assignment. The published assignments are now visible on the student's dashboard. The student can watch the A.I. generated animated video or read the assignment text. To test their knowledge, they can take the quiz for the assignment on the portal, upon completion they get a score of how well they performed. Once the deadline for the assignment is reached the teachers can view the results for the assignment, to see how well the students performed. Using TypingDNA ensures that the students don't indulge in malpractices like exchanging passwords and also serves as an efficient remote proctoring tool.

How I built it

Izzy is built on top of the MERN stack. The frontend UI is built using Material UI & ReactJs. Redux is used for state management in the application. The backend is built using Nodejs for the learning management system and flask deploying for the proprietary video generation algorithm. The authentication is implemented by using TypingDnaRecorder-JavaScript on the frontend, specifically the .addTarget() & .getTypingPattern() methods to implement type 1 (sametext pattern) authentication. The backend uses the TypingDnaClient-Nodejs's .check() & .save() methods to login and register users respectively. The grades, courses, classrooms, students, teachers, assignments and results are stored on a hosted PostgreSQL database. And the video generation and question generation requests being CPU intensive tasks are done by different servers. The video generation is explained in detail. We use typescript to reduce the number of errors.

Challenges I ran into

  • Using TypingDNA's Javascript Recorder with React's virtual dom
  • Making the UI as smooth as possible for the students and teachers
  • We had to choose the right color themes and templates for creating AI powered video's
  • Using version control on multiple repositories

Accomplishments that I'm proud of

Izzy completely changes the way, Ed-tech is seen and takes the concept of visual learning to the next level. Ed-tech startups spend a lot of time and effort to make quality video content for their platforms, with Izzy doing this is a piece of cake. Despite being a team of 4 twenty year olds we were able to build a fully functioning learning management system and deploy it. We were able to create a first-of-kind video based ML algorithm that takes any wikipedia article, summarises it and makes a video out of it within minutes. (10 minutes max)

What I learned

  1. Using TypingDNA for keystroke dynamics authentication.
  2. Making API using TypingDNA Nodejs client
  3. Setting up a fullstack MERN application
  4. Using advanced Natural Language Processing techniques to automatically generate questions from any wikipedia text
  5. Managing multiple codebases
  6. Setting up a REST API with Nodejs
  7. Setting up a REST API using Flask and Python
  8. Deployment to Digital Ocean.
  9. Using Premier Pro to edit videos like a pro ;)

What's next for Izzy Learning Management System

  • Improving our proprietary video creation algorithms to support multiple languages like Hindi(हिन्दी) Chinese (中文) or Spanish(español) to name a few
  • Implementing typing support for phones and on native devices
  • We plan to add a chat bot for doubt solving
  • An Alexa skill that supplements the quiz taking process is also under development.
  • We also plan to add text recognition to convert images containing long and boring texts to videos

Reference

https://blog.typingdna.com/wp-content/uploads/2020/05/typingdna-elearning-whitepaper.pdf

+ 5 more
Share this project:

Updates