Math students are faced with a dilemma: how to take notes in class? Transcribing equations in digital notes takes forever; doing it in class makes it hard to pay attention to the lecture, and doing it later can often lead to hours of time spent formatting. But having neat, indexed notes makes studying easier. What's the solution?
What it does
BoardTex allows students to snap a picture of a whiteboard and have it converted to LaTeX, equations and all. Users can tag pictures for convenient access later.
How we built it
We used Google Cloud's OCR to handle character recognition, then parsed the raw character data into LaTeX code using Python (with the support of pillow, json, and several other libraries), along with the MathPix API.
Challenges we ran into
Integrating the different parts of our project was a real challenge. User interface, character recognition, text parsing, equation detecting, and more all had to fit together seamlessly, which gave us no end of trouble. We also spent several hours on blind alleys with various neural networks before settling on using MathPix.
Accomplishments that we're proud of
We had no idea what we wanted to do until midnight on Friday evening and didn't get a serious start until late Saturday morning, but we still think we produced a great project. BoardTeX is something that we're excited to try out in our classes. We hope that it becomes a useful tool for math students everywhere.
What we learned
It's difficult getting everything to fit together. Most of our issues arose when synthesizing different parts of our project. We also learned that when you're working to a deadline, it's best to get something up and running, even if it's not perfect. We also discovered that React is a steep learning curve.
What's next for BoardTeX
A web interface would be the obvious next step. We want to improve the capabilities of BoardTeX: ideally, we want it to handle an entire blackboard in one photo, instead of theorem-by-theorem. We'd also like to expand to disciplines other than math; our current setup relies on the specific format of math board presentation. A feature that allows users to specify their own macros for the output LaTeX code would be very useful.