-
-
The landing page for scribbletex
-
The logo for scribbletex!
-
A simple example
Inspiration
Our group's initial inspiration stemmed from the monotonous task of continuously searching for complex LaTeX commands online. We could easily write down any of the mathematical equations on paper with ease and in a timely manner. This year, during our machine learning class, we finally solved our problem. The course's knowledge and tools allowed us to swiftly convert drawn mathematical equations to LaTeX code.
What it does
The project takes input from a user via a canvas that they can draw mathematical functions on, which is then converted to latex code and the respective rendered latex text through a generative large language model.
How we built it
Firstly, we constructed the canvas for the user to draw and scribble down any desired equation to be converted. Next, we needed the canvas and the generative LLM to communicate somehow. We recognized that the canvas can be converted into an image and then a base64 string. This string is then passed to an API call which communicates with the LLM in order to generate the LaTeX code. Once the LLM produces the desired response, we parse the output to ensure the validity of the response shown to the user. Finally, the parsed response is the desired result from the sketch provided by the user, which is then rendered to the web application.
Challenges we ran into
Our group wanted to create our own nerual network for scribbletex, training, validating, and testing it. However, with the two day time frame, we believed that this was too ambitious. We ultimately resorted to using a generative LLM that already existed.
Accomplishments that we're proud of
We take great pride in completing this project, taking on numerous ambitious tasks collaboratively. Our group provided support and guidance to each other throughout the entire process, resulting in scribbletex, a project we are excited to showcase.
What we learned
How to utilize a generative LLM and communicate with it via API calls. Some of our group members had no prior experience with the React framework but developed a strong foundation for the framework. Additionally, we learned how to communicate among one another, along with compromising and making progress towards scribbletex.
What's next for scribbletex
For scribbletex, we would to implement new features, which include a user history to store previously queried mathematical equations and their respective output. Additionally, we would like for users to upload their own images for translation to latex as well!
Log in or sign up for Devpost to join the conversation.