StudySync

StudySync
Home Page
Joining Room
Room Setup
Quiz
Detecting Sleeping

Inspiration

Who doesn't love studying with their friends; getting work done with countless hours of fun conversations. But when you have a looming midterm the next morning, maybe you shouldn't study with your friends?

That was until StudySync, a collaborative studying platform that optimizes your ability to study with your friends. By tracking attention and note taking using deep learning, users get assigned points for focusing with a leaderboard of everyone's score at the study session, allowing you to stay motivated to study and encourage your friends to not fall behind. With AI detection when users are going on their phone or about to sleep, StudySync uses AI voices to wake users up and yells at users to put their phone away.

StudySync understands the importance of active recall, and by inputting your notes/course text, StudySync will periodically give a quizzes based on content the group just finished studying. Our smart quiz system pairs you with peers to master difficult topics and lock in. With public and private study rooms for introverts and extroverts alike, our app creates a vibrant, supportive environment for learning—anywhere, anytime for a diverse group of people, thus, promoting diversity in educational attainment. However, StudySync knows the importance of breaks as well, and gives the group periodic breaks.

What it does

StudySync isn't just another study app—it’s your new go-to for leveling up your learning game with a dose of fun. Imagine hoping into a collaborative study session with friends, where your attention is tracked by your camera using OpenCV and MediaPipe. Stay locked in and score points, but if you start scrolling through your phone, brace yourself—your points take a hit 😲. And for an extra twist, you'll get a dramatic cutscene with a voiceover (thanks, ElevenLabs!) calling you out, keeping things lighthearted but on track. The app can even tell when you're just plain exhausted, suggesting a break or nap when you need it most, with built-in Pomodoro timers for structured rest. When it's time to test your knowledge, StudySync serves up periodic quizzes based on the material you’ve uploaded, and the app toggles between quizzes and time intervals as you complete sections. Active recall keeps the pressure on, tossing in questions from earlier sections, while the app keeps track of your mistakes, pairing you with peers who can help fill in the gaps. It’s not just studying—it’s an interactive, engaging learning experience designed to keep you sharp and connected.

You're also able to create public/private study rooms – helpful for people who are introverted and want a study partner, and implement a system where you can input how many days you have before a test/quiz—it'll tailor your study plan to help you lock in on the topics that matter the most!

How we built it

We built the backend with Node.js, Express.js, and used WebSockets for real-time features like room creation, joining rooms, sending messages and updating points. We also used HTTP (REST API) to handle regular POST requests with the Express framework. These requests return JSON data about rooms and their existence or provide a basic welcome message for users interacting with the backend directly, and is hosted on Vercel. We used other features to make the computer vision task work, such as OpenCV, and MediaPipe for facial detection and a DNN using TensorFlow, and for object detection we used a CNN dataset trained on 9,900 images. We also used ElevenLabs' Text-to-Speech API for a much more expressive voice.

The frontend is primarily built using JavaScript/TypeScript with React and Tailwind CSS. Socket.IO is used for real-time communication between the frontend and backend. For webcam access we used WebRTC, then we used a Fetch API for sending captured frames to the backend for attention analysis. We also created a leaderboard system to add in our gamification effect.

Not only that, but to keep track of user and session data, we implemented OAuth authentication using Clerk!

Challenges we ran into

Firstly, we didn't have full team to begin with, as well as all our members having midterms during the time of the hackathon, in particular, one member in our team has her midterm on the second day of the hackathon, which was a difficult challenge as we navigate through how to make the best use of our time. Throughout this weekend, it has been essential to balance working on our project with our mental health, rest, and last-minute study. 2 of our members were also from Waterloo, so commuting back and forth was a difficult challenge for us. As we work in a team of three, we had to decide on a project idea, scope, features and get working on it immediately due to the limited time given the challenges above. Maximizing our productivity was difficult when some tasks depended on others.

We encountered challenges with merging Git commits, often overwriting each other's code and introducing bugs. To overcome this, we had to learn new technologies, techniques, and adapt quickly, and learned new technologies to make our ideas happen such as navigating around the computer vision aspect. We initially started with a variety of features we wanted to work with, and ran into a challenge where we had to decide which to pursue and how to pursue each individual features.

To navigate these challenges, we read documentations, used Stack Overflow, and spoke with sponsors and mentors. With some turnarounds, we solved them and executed out the features (mostly) according to plan!

Accomplishments that we're proud of

We’re proud of the significant progress we’ve made in such a short time on a project we initially thought was overly ambitious. We've successfully reached the majority of our key milestones and have also picked up and learned different tools along the way, such as navigating around the computer vision aspect.

What we learned

Tackling computer vision and integrating tools like OpenCV and MediaPipe introduced us to a host of new technical challenges. With that said, we learned to quickly adapt, troubleshoot, and leverage documentation and mentorship. Tackling CV and integrating different features with tools like ElevenLabs introduced us to a host of new technical challenges, but we learned to quickly adapt and troubleshoot.

What's next for StudySync

StudySync is just getting started. Our next steps are focused on enhancing engagement and personalization. We’re working on new ways to customize study plans based on individual needs, such as tailoring quizzes and attention tracking based on how close exams are, indicated by users. We’ll also be expanding our social features, making it easier for introverted users to join public or private study rooms. Integrating active recall, cumulative quizzes, and collaborative teaching moments will ensure deeper learning and concept mastery. Plus, refining attention tracking is another major next step, with more advanced metrics to help users manage fatigue and focus. We hope to incorporate more features like uploadable materials, interactive cut scenes, and real-time study performance insights. Stay tuned for the next wave of immersive, collaborative, effective study experiences!