Study Buddy

Inspiring attention through expertise in Psychology, EdTech and AI.

Inspiration

When you are studying for an exam (or competing in a hackathon!) it can be easy to lose motivation. Maybe you tune out for a second and check your phone or start to nod off. Half an hour later you realize you haven't made any progress. This can happen without you even realizing what's going on until it's too late. Not anymore! Study Buddy will notice you starting to waver and help get you back on track.

What it does

Study Buddy accesses your webcam while you study and, with the power of AI, can detect when you look away from your computer. If you do, Study Buddy will gently alert you to return to work. As you continue to disobey, the tones progress through our team's proprietary ABC ("Always Be Concentrating") auditory stimulus sequence. This sequence has been carefully calibrated, by our team of psychology and education experts, to maximize your desire to return to work. Once you re-engage, the tones will stop. At the end of your study session, you receive information about your attentional states in order to facilitate future improvement.

How we built it

Study Buddy is built on top of fastai, which allows you to build neural nets quickly. We used a CNN with a resnet34 architecture that was pretrained on the ImageNet dataset. We then tuned our model with training data collected during the hackathon in the Theatre Space. OpenCV is used to pull webcam data during live study sessions. Every second, an image is tested by our model and the studier is classified as engaged or not engaged. When a studier is determined to be looking away from the screen for too long, pygame and music21 are used to play the tones. Gooey provides a clean way to retrieve study session lengths from our users and matplotlib is used to produce the final statistics.

Challenges we ran into/lessons learned

Study Buddy works well with new people, but can struggle if your face is not easily visible. Faces need to be centered for the webcam when working.
It's surprisingly complicated to create and play custom sounds in Python without using external APIs (which would compromise portability).
FastAI and non-Linux OSes are not friends.

Future improvements

Further sampling will be needed to ensure the robustness of our model against varied environments and face visibility.
There remain untapped motivational opportunities that can be leveraged using alternative sensory stimuli.