DigiNotes: Classroom Notes Assistant

DigiNotes converts handwritten notes from several students into text format which is analyzed to give a detailed summary. The program outputs a pdf with all of the students notes and the summary.

Comment

Inspiration

As college students we have often had struggles collaborating with our peers due to a lack of a shared information, difficult to read hand writing, and limited time.

What it does

We created DigiNotes to make an efficient way to compile hand-written notes from several students into an easy to read pdf with formatted text. The program prompts the user to input an image file which, after preprocessing to make easier to read, is read by Gemini to convert the handwritten and printed text into formatted typed text. The new text is sent to LaTeX which converts it to a pdf.

How we built it

We used a combination of ai models and image editing. Through image editing we gray scaled it and improved its contrast. We then used ai to convert the text into latex and turn it into a pdf version.

Challenges we ran into

The first three major problems we ran into was formatting and losing information from text extraction. The more we tried to fix the formatting problem through ai prompting the worse the integrity of data extraction would become. So we went to the mentors for help. They recommended we try other models aswell as try to produce more consistant results. Half our team decided to explore different ai models instead of gemini, while the other half went to try to create consistant results. The different ai models didn't go well because several of them were behind paywall. Through a long process of looking for documentation we learned that a different gemini model as well as the parameters seed and temperature would produce consitant results. Afterwards we went to fix the loss of integrity of information. We used cv2 to grayscale our images and improve the contrast so that gemini would have a easier time processing it. It improved our results but there was still data loss. Then a mentor suggested we use gemini to produce latex code instead of text extraction. This caused an increase in our data integrity.

Accomplishments that we're proud of

We are proud of how much progress we have made in the short time provided and the general completeness of the project. We are especially proud we successfully uses Gemini AI to accurately convert handwritten notes, including some charts and diagrams, to typed text.

What we learned

We learned more about ocr and how it works behind the scene. Also on how images are read and how features such as blur work. We also learned about ai and latex code in general.

What's next for DigiNotes

The next thing we are going to focus on for DigiNotes is implementing the AI summary of all inputted notes. We could further improve that in the future by allowing adding sections to the notes and the AI could summarize each section independently. We also want to refine the prompt given to the AI to allow understanding of flowcharts or other difficult diagrams and to exclude crossed out words.

Share this project:

Updates