PictoPlan

PictoBuddy :)

Inspiration

One of our team member's parents is a teacher, so they know firsthand how time consuming creating lesson plans for students can be. A teacher should be focusing on building connections with students and educating them, not laboring over course content. We've all experienced fantastic teachers who truly made a change on our educational experience, and we hope that PictoPlan can shoulder some of their burden so they can focus on what really matters.

What It Does

PictoPlan lets the user select their own photos of content from books or textbooks and upload them to our site. After this happens, the photo is converted into text using OCR and then passed through OpenAI's API in order to summarize and create a lesson plan for the respective content. This lesson plan is then displayed on the website, and is accessible through a permanent link to facilitate teachers sharing lesson-plans and collaborating.

How We Built It

We built the backend primarily through Python and FastAPI, where we wrote most of our prompt and Pytesseract-OCR related code. We also wrote our MongoDB API's and our routing through FastAPI as well. For frontend, we used Flask and we deployed our services on the cloud using Heroku and Cloudflare at pictoplan.org!

Challenges We Ran Into

As this was the first hackathon for 3/4 of us, we ran into a lot of technical challenges.

New Stack: It was our first time working with this tech stack, so we certainly had our work cut out for us and spent a long time familiarizing ourselves with it.
OCR: A problem we ran into was ensuring readability of the images for our text generation. This was solved through tinkering with image pre-processing and comparing results.
Deploying: Handling the networking for deploying on cloud was a major issue we had to tackle and it took time to verify that our routing and networking of components worked properly.

Accomplishments That We're Proud Of

On an individual level and as a team we're extremely proud of what we were able to achieve. It was our first time using any OCR, so to be able to accurately pull in text from images was an awesome accomplishment. Additionally we haven't worked with OpenAI's API before, so we had to spend a lot of time adjusting it before we were happy with the results. As we're all close friends so it was very fulfilling to watch each other tackle problems and come out a tad bit smarter, so I'd definitely say the biggest accomplishment was pushing through the struggle and creating something we're proud of.

What We Learned

Individually we all learned a lot about each technology we used, as it was essentially our first time being exposed to it. Beyond the minutia though, we learned a lot about what it takes to develop as a team, and many soft skills. We've all worked in small teams for projects before but not really under a time crunch, and while pressure does make diamonds, it also makes a lot of stress. So we learned a lot about effective communication, time-management, and collaboration.

What's Next For PictoPlan

We're really passionate about this project and really think it could make a difference to educators, so we have many ideas for how we want to continue this after the hackathon. We'd like to improve our image pre-processing to allow our OCR to better collect text from poor quality images. We would also like to further develop the front-end to make it more user friendly. We want to make the website as accessible as possible for teachers in terms of interfacing and experience, and that is something that must come with time.