As most of us are students, we were really interested in improving the quality of online education, especially in such an extreme time.
What it does
The platform analyzes a student's behavior during an online lecture and leads through the educational materials in the most easy-to-understand and productive way. The AI team analyzes how attentive and interested the student is, using only a webcam for further recommendations to increase the learning efficiency.
How I built it
We used open-sourced light-weight deep learning models for both body pose estimation and 3D head pose estimation. Then we built a tiny network on top of the outputs from previous models, and trained the network to classify the student's current state. All models worked in real-time on CPU so that we could deploy one of them into a web, connecting with front-end part.
Challenges I ran into
Most open-sourced solutions were difficult to reproduce. Especially in the case when we wanted to run different networks in a single pipeline. We had to start from the beginning several times because of the inconsistencies between models. And at the end we could find out and set up such solutions that fitted our goals perfectly (could run in real-time and were robust enough) and worked together under the same environment.
Accomplishments that I'm proud of
We made a small research on how much visual features are correlated with attentiveness, and which features are most important for it.
What I learned
Building up working solution in a short amount of time and being able to iterate through different approaches.
What's next for YouLearn-AI
YouLearn-AI is better than attending university.