VisualIQSolver

image to be converted to Latex
Latex conversion after running through file upload

Inspiration

We were inspired by the struggles of college students in classes like math or physics. Word problems can be very difficult and challenging to approach. So we decided to create an app where you can scan the image and have your problem solved by ai.

What it does

The program parses an image(handwritten or typed) and converts the text to LaTex format, which runs through OpenAI to give a step-by-step solution to the problem.

How we built it

We initially split into 3 different parts. The first part used Tesseract in Python to parse and scan the image and convert the contain into LaTex. The next part was to use OpenAI to answer the questions in LaTex and give the steps and answers. The third part that is still in the work is to have a working mobile app made using Flutter/Dart compatible with IOS with the ability to take a picture, and save and store the picture, and parse it.

Challenges we ran into

We ran into many challenges regarding the project. To name a couple, we ran into problems with VS code and importing libraries. We also had to change direction of our app because we wanted to be able to have a camera to capture a picture and read it, but it was deemed to be too difficult integrating everything together within the 12 hour time constraint. This was also all of our first time using Tesseract, which is an open-source OCR engine that allows us to extract the text from an image. We ran into a lot of issues like it wouldn't scan some images or it wasn't able to get whole expressions sometimes. The compatibilities between mac and windows also made this very difficult.

Accomplishments that we're proud of

We are proud of how we managed to successful get text from an image the user uploads and change it into latex code. We are also proud of how we managed to create an AI chatbot that can answer questions like what 3+3 is or what the largest planet in the solar system is.

What we learned

We all learnt a lot, like for example about OCR engines, ai, and flet. We learned how to work together using GitHub since it was all of our first times working together and about app development/phone plugin integration with flutter . Open ai and image to text extraction were also some interesting topics we dove into while working on this project.

What's next for VisualIQSolver

Finish developing our part of getting access to the camera and storing it. Then using the picture we took to run in the tesseract part and get the latex text to input the output to ai to get the steps/ answers. Also finding better ways for syncing the code from Windows to Mac.