A widespread problem for university students in our class is that our professors write too quickly on the blackboard while talking and it is impossible to copy everything down and get a good understanding of the material at the same time. To solve this issue, we built a platform that takes care of the note taking for you - all you need is a picture of the board to have neat notes transcribed for you in Google Docs like format.

What it does

BoardScribe is a classroom collaboration social network environment that uses image recognition to transcribe blackboard notes from a photo. It allows for notes to be uploaded and transcribed, but that's just the first step. Once notes are uploaded students and teachers can make comments to clarify things and ask questions.

How we built it

The image recognition and machine learning were handled by Tessaract.js and Clarifai respectively. We were able to get pictures of individual characters out of pictures of text and pass those into Clarifai to transcribe into characters. We trained the AI extensively with over 1000 handwriting samples of different characters and continue to pass through different inputs to improve it day-by-day. For the platform we used the Django web framework and the SQLite database to store user's notes and group notes by class. Students also can comment on each note with suggestions and corrections using their facebook accounts

Challenges we ran into

The Clarifai API took a while for us to get the hang of. Building the platform took many iterations.

Accomplishments that we're proud of

We got the image recognition down.

What we learned

We learned a lot more about machine learning and image recognition.

What's next for BoardScribe

We will continue to develop a MVP and also potentially include the MYO Armband into our design to track chalk/marker movement.

Share this project: