-
-
Main dashboard and the LockIn team.
Inspiration
Our inspiration and goal with this project is to help college students be more effective in their academic learning. With the ever increasing pace of innovation and research, its important for students to optimize their learning so that they can pursue external projects or interests that will enhance their preparation for their future careers.
What it does
By keeping track of what the professor is saying, the lecture slides, and the engagement of the student during class, our pipeline can evaluate the engagement of the student and guide him to the key ideas and concepts that were missed during lecture.
How we built it
We built it using react, python, and Hume's expressive API. Specifically, we utilized react to create a dashboard UI for the students to upload their lecture slides and start the program. For the backend, we made a server using Flask. In the server, we processed the input slides, the responses from Hume's API, and the audio recordings.
Challenges we ran into
The most challenging part was identifying a method to map the time intervals in which the student was not engaged to the corresponding part in the lecture audio file. Another challenge was processing the responses from Hume's expressive API to a format that allowed us to better identify student engagement .
Accomplishments that we're proud of
Mapping the time intervals where students were not engaged to the corresponding timestamps in the audio file, tuning Hume's expressive API responses to better identify student engagement, and implementing an agent to create study guides and strategies for the student based on the key ideas and concepts that were missed.
What we learned
How to create a python backend server, how to setup a Websocket connection using javascript, and how to create an LLM based agent.
What's next for Cognitive
Enhance the current engagement model to be more robust by considering other expressions and facial cues that are not indicative of engagement, create a professor interface to monitor the level of engagement displayed by the students during class and allow them to their pedagogical techniques based on data. Moreover, enhance security features for commercialization.
Log in or sign up for Devpost to join the conversation.