EnGauge

Inspiration

In the age of virtual classes, we often hear our professors complain about missing the real-time feedback of face-to-face lectures. For this hackathon, we wanted to solve this using some sort of AI. After looking through Google Cloud, Vision AI seemed like the perfect fit.

What it does

EnGauge connects students and professors in a video-chat room using WebRTC. This peer-to-peer network is responsible for handling the video stream which is used to analyze student's emotions. When the professor clicks the 'Gauge Reactions' button, screenshots of the students are sent to Vision AI to analyze their reactions. This data is then displayed to the professor so they can assess class engagement.

How we built it

Our development process started out very segmented, with each team member having a specific area they'd focus on. The entire thing is hosted and deployed through Google Cloud Platform. Our codebase is split into two halves: back-end and front-end. The front end is deployed into Google Storage buckets with Load Balancing using Cloud Build, and the backend is hosted on serverless Cloud Run which we achieve by dockerizing our server in Cloud Build and deploying. The second half of the hackathon was spent working together on configuring WebRTC for video streaming and getting Vision AI to analyze screenshots of students.

Challenges we ran into

One big challenge we faced was setting up our CI/CD deployment pipeline. No one on our team had previous experience with Google Cloud Build, so getting that running was a bottleneck on testing.

Accomplishments that we're proud of

What we're most proud of is the fact we were able to solve a common problem using cool technology. We believe our solution is a creative way to solve a very modern problem. We tried a lot of new things during this hackathon and are very happy with the new technologies we learned.

What we learned

One of the lessons we learned from this project is that it's ok to cut some corners when on a limited timeline. While a lot of the tech we used was a big inspiration for us, we could've saved some time by going more traditional routes. One example of this is the CI/CD pipeline, which we're very happy we learned how to use, but considerably increased our development time.

What's next for EnGauge

There's multiple features we thought of during brainstorming that we unfortunately didn't have time for. For these stretch goals, we'd like to take a couple pointers from Zoom's playbook. Most notably, we all wanted to make a screen share feature and would like to add it in the future. Additionally, our team plans to go to VTHacks in a few weeks and apply the lessons we learned on this project.

Built With

• firestore
• flask
• google-cloud
• google-vision-ai
• node.js
• react
• webrtc