Polarity

• 

  Polarity

Inspiration

Our project, Polarity, was born in the midst of an educational crisis, where overwhelmed educators are often unable to provide written notes for their lectures. This not only places a significant burden on students to accurately document these lectures but also takes away valuable time that could be spent on understanding and internalizing the material. We saw a unique opportunity to use technology to fill this gap by creating a tool that not only aids students in generating their own precise notes effortlessly but also empowers teachers.

What it does

With Polarity, educators can record their teachings, and the system will automatically transcribe and convert these recordings into detailed, well-formatted LaTeX notes, with every mathematical equation and concept captured accurately from voice to paper. This approach not only ensures students have access to high-quality notes but also streamlines the note-distribution process, making it more efficient for teachers to provide essential study materials to their students.

How we built it

We first created a dataset of LaTeX files manually based on audio clips from a variety different math and physics lectures, in order to maintain a more representative data sample. After that, we fine-tuned our model utilizing data pairs of audio clip transcripts (from speech to text) to LaTeX code. We turned to establish our formal data pipeline at this point, where we utilized MonsterAPI speech->text and then GPT4 text->LaTeX.

Since the data was trained, and the pipeline was fully working at this point, the only task left to tackle was building a website using the innovative Reflex.dev framework. We were able to do this and connect it to our API chain (MonsterAPI/GPT4), finalizing our work.

Challenges we ran into

One of the biggest challenges we faced was dealing with model limitations earlier in the hackathon. We initially tried to train a PredictionGuard suggested model, DeepSeek, on our task. However, we faced limitations with the token response being very low for these models. After switching to GPT4, nearly none of these issues were relevant anymore.

The next major issue we had was the efficiency of the model - it became too slow to the point of not being able to function properly, causing runtime errors with our environment. We had to make several changes to make the model faster, including changing our prompts and our data pipeline.

Accomplishments that we're proud of

Polarity’s capability to intelligently organize lecture content into a structured format stands out remarkably. Not only does it transcribe words but it also discerns the flow of the lecture to create meaningful headers and sections autonomously.

Beyond that, the accuracy with which Polarity renders mathematical equations, symbols, and concepts is unparalleled. Through rigorous training, our system has mastered the complex language of mathematics, allowing it to replicate the exact equations and symbols discussed in lectures. This capability ensures that every notation, from simple algebraic expressions to intricate calculus, is flawlessly represented in LaTeX (textbook) format.

Polarity is also equipped with advanced speech processing algorithms that filter out filler words commonly found in spoken language. This ensures that the final notes are concise and free of clutter, containing only the critical information that students need to focus on. This feature is especially beneficial in maintaining the high quality and readability of the notes, ensuring they serve as an effective study tool.

What's next for Polarity

We plan to roll out Polarity as a product for schools and universities across America, hopefully easing a burden off of overworked educators by providing a method to generate class notes for students with ease.

While many AI models already exist for general information, there is not a currently feasible model that can specifically help implement work in LaTeX. We are working to create an API that allows educators to specifically tailor a AI model to their class structure and format.

Built With

• gpt4
• monster.api
• python
• reflex
• wispr.ai