Blackboard

Inspiration

Existing online whiteboards do not fully satisfy our requirements and are not responsive enough, so we set out to create our own whiteboard solution that is fast, customizable and lightweight. We decided to theme our whiteboard as a blackboard as an homage to the legacy of sharing knowledge in traditional classroom environments.

What it does

Our project facilitates collaboration on our online whiteboard, with functionality to convert formulae to LaTeX, bridging the gap between handwritten notes and a more formal format, and providing accessibility to create LaTeX intuitively.

How we built it

This is written as a Next.js app from scratch, for complete control. Clientside WebSockets interact with a separate Node.js relay server for real time collaboration, which employs proprietary communication protocols. A Python script is invoked by the hosting server to power the AI conversion; it functions as an API that takes the image as input and returns the LaTeX as output (somehow).

Challenges we ran into

We originally planned to support converting the entire whiteboard to LaTeX, but struggled to find an accurate semantic segmentation model, computer vision technique, or multi-modal transformer to split the document and extract layout information. We reduced scope to involve converting individual formulae and built a user tool to select what needed to be converted, shifting the onus onto the user to choose what should be converted.

Accomplishments that we're proud of

The real-time collaboration is responsive, the design language is unique, and it has useful applications in the real world.

What we learned

Next.js, WebSockets, Hugging Face transformers

What's next for Blackboard

More functionality, possibly tuning or training a model to improve LaTeX conversion, improving smoothness and error handling of communications, increasing robustness of synchronicity between users.