EduVisor

Homepage
Sample dashboard for a lecture
Previous analyses

Inspiration

As college students, we can all relate to sitting in lectures where we’ve felt lost. But, more often than not, it’s not about the content, but about the professor’s delivery. While it’s easy to point fingers at educators, a core issue is the lack of regular feedback to help them refine their teaching strategies.

Traditionally at Georgia Tech, the only feedback they receive is through CIOS, which happens at the end of the semester. This system doesn’t provide for real-time insights or mid-course corrections.

We believe that with regular, constructive feedback, educators can adapt and evolve, ensuring a more engaging learning experience for students

What it does

With our application, EduVisor, educators have the opportunity to consistently submit their lectures for analysis. Our sophisticated AI-driven evaluation mechanism analyzes both voice & transcript, and offers personalized insights into their teaching methods. By assessing aspects like speech emotions, tone modulation, class engagement, and voice pace, we ensure that teachers can truly resonate with and captivate their students. We also leverage Large Language Models to provide the necessary recommendations to teachers.

How we built it

The application is built with the Django web framework for the backend and HTML, CSS and JavaScript for the front-end. The lecture videos that were uploaded were split into chunks of 30 seconds using Python's pydub library. These chunks, or utterances, were first fed into the wav2vec speech emotion recognition model through Hugging Face, and then transcribed through Google Cloud Platform's Speech-to-Text model. We performed various operations on these processed utterances to extract our insights. This was finally fed into the GPT-4 API in order to generate recommendations for the educators. The application was connected to the Django ORM, which is Django's built-in database functionality that is operated through MySQL.

Challenges we ran into

Deploying ML Models: We were initially facing challenges deploying our machine learning models efficiently through the application. We chose to use GCP's Speech-to-Text API in order to make this process more efficient, and made it such that our Hugging Face model was only loaded once at a time.

Integrating front-end with Django: One of the challenges we ran into was that we made some of our html pages outside of our Django, and when trying to integrate them into Django once completed was a challenge, especially with Forms and Upload buttons.

Front-end: Most of our teammates didn’t have much experience with front end development, so getting all the webpages running, and integrated was very time consuming.

Fine-tuning LLM: We used an API key into a GPT’s LLM model to help create personalized feedback based on our analytic metrics. However, fine tuning the results and finding the perfect prompt proved to be a challenge.

Accomplishments that we're proud of

GCP and HuggingFace:While our team members had previously worked with machine learning models, none of us had deployed them for production. We were extremely happy to take advantage of Google Cloud Platform and Hugging Face and deploy machine learning models efficiently.

Front end: One of the things we’re proud of is that none of our team-members had much experience with front-end development, but we were able to take some crash courses, and build a comprehensive, well-designed front end for our application

Creativity and Innovation: We are proud of the creativity we demonstrated throughout this hackathon, from ideation to development. In particular, we are proud of the various research based metrics we were able to derive from the voice and speech transcript, like tone modulation, engagement, and words per minute, that can help a teacher improve their lecture.

Impact: We've developed a tool that enables educators to assess the engagement level of their recorded lectures. This innovation goes beyond traditional educational technology, providing valuable insights that help teachers enhance their teaching methods.

What we learned

Django Development: We dived into Django, mastering its Model View Controller structure, authentication, and database management through tutorials and hands-on practice. Our dedication led to a secure, scalable platform for EduVisor.

Front-End Development: Despite our limited experience, we honed HTML, CSS, JavaScript, Tailwind & React skills to craft EduVisor's visually appealing and user-friendly interface.

Google Cloud Platform (GCP): Exploring GCP, we harnessed its potential for enhanced performance, scalability, and efficiency, aligning with EduVisor's vision.

Machine Learning Models: EduVisor's success relied on Hugging Face and OpenAI models (DaVinci-003 & GPT 3.5-turbo). We navigated complex tasks like data preprocessing and model training.

What's next for EduViser

Fine-Tuning Speech Emotion Recognition:Continuously improving speech emotion recognition algorithms tailored specifically to the needs of teachers and classroom environments. This will enhance the accuracy of engagement assessments.

Student Feedback Portal:Creating a student feedback portal within the application. This feature will allow students to provide valuable input on lectures, fostering better teacher-student communication and improving learning outcomes.

Scalability and Accessibility:Ensuring that EduVisor can accommodate a growing user base and remains accessible to educators with varying levels of technical expertise. Ideally we want to be able to import lectures from Canvas LMS or DropBox.

User Engagement and Feedback:Actively engaging with users to gather feedback, understand their evolving needs, and continuously refine the application based on real-world usage.