Due to covid 19, everything went online, even school. We never imagined that studying could happen anywhere else than school! The concept of distance learning is revolutionary. It has its own pros and cons. While attending classes from the comfort of your home you can get distracted very easily. No one supervises you virtually, in physical classrooms there is one to one interaction so people are more focused but virtually we tend to get distractedted
Yes, Distraction is global, you are not the only one facing it. 70% of MIT students say that studying distant from school is difficult. That's why we made an app that increases your focusing power. Like when you have the feeling that someone is watching you, you ought to work better :D
FOCUS has entered the chat
What it does
FOCUS is an ML-based full-fledged web application that is all set to increase your focus. You just need a webcam!
The web app is capable of monitoring multiple aspects including
- Sleepiness meter (yawning, head bending)
- Behaviour meter (restlessness, looking away, movement)
- Posture (Rates your posture, even suggests you)
- Attentiveness (Facial detection,)�
- Overall focus score (using factors to determine overall score) It detects all of these things and provides the user real-time data about it You also get different types of alerts for different cases which suggest tips to you. It tells you about your posture and how you can improve it.
How we built it
Building this was a complex process. We used various things accompanied and polished by our skillset
ML Model - The Ml model was a really complex thing. It just didn't detect one thing, it detected 3 things (Facial recognition, posture recognition, Hand detection,) We compiled all the datasets to form one big dataset and then made the model. We used several libraries as well
- We used TensorFlow to Build our model.
- Open CV library for facial detection
- Create a classification model for detecting behaviour using mobile net
- We finetuned it on imagenet weights
- For sleepiness behaviour, we use haar cascade classifier for detecting face and haar cascade eye detector for detecting images
Rest API - For connecting the frontend to the ml model we needed a REST API. We coded it using Flask and python. The workflow of our Rest API was like the app captures frames every 10 seconds and then the image is converted into base64 form and the API sends it to the ML model which provides prediction and the values are updated on the frontend.
Compiling all of these is what makes our app
Challenges we ran into
OMG, we faced some very very enormously hard challenges, of which some took hours to solve. I will list them
- Image with the REST API - Transfer of text and other binary data is very very easy with a REST API. But image transfer is very painful. It took us 3-4 hours to solve it. Firstly we tried to store it in the memory and it didn't work, the image wouldn't be transferred. Then we tried to save it in local storage then the ML model couldn't process the Image. Finally we stored it locally on our desktop and send it to the model. The image was updated/overwrote every 10 seconds which generated continuous new responses from the ML model. If we had decreased the time interval from 10 to any less the API would ve crashed.
2, Lack of dataset - For our work, there was no specific dataset. That's why we needed to create our own dataset. We processed over 30GB of images which took like 2 hours. Training the ML model was a big task too. Very happy to say we achieved 93% accuracy
- Time management/Timezone differences - Time management was a big challenge everything was very messy at the start and due to timezone differences, we were not able to attend workshops :(
Accomplishments that we're proud of
We are really proud of
- Actually thinking of a project related to distance, we interpreted distance in form of online distance learning.
- Getting 93% Accuracy on ML model
- Implementing a REST API server
- Creating our own dataset.
- Successfully integrating webcam component using an npm package
- Making use of as much as existing technologies we could
What we learned
Everyone had different learnings from this project. I will sum up them all -
- A deep knowledge of REST API
- Making our own dataset with various classes
- Using react packages
- Integrating a real-time webcam that captures images after a given time interval
- Teamworking skills
What's next for FOCUS
We have many things in mind for it. As we had limited time many features we had planned but did not integrate
- Having a point/rewards system
- Authentication system
- Deploy it online
It was a very fun experience for us to make this project. All code we wrote by ourselves