Omni: Screenshot to Calendar

Inspiration

As students at Purdue, we deal with a lot of scheduling and it can make planning stressful. We all know the feeling of missing a meeting and the anxiety it brings. Our goal was to build a tool to help reduce stress in students across the country. Did your schedule just come out? Take a screen shot and we take care of everything for you. Did your teacher post office hours on ed discussion at 1 AM? We've got you covered with a simple chrome extension. We offer a seamless platform where you can give us a screenshot of any event related text, and we can manage that and update your Google Calendar. Omni: planning made easy.

What it does

We prioritized ease of use with our produce. Therefore, we leverage Google Extensions so users can take a screenshot of event text at any time on their browser and leave the logistics to us. We set up google calendar events for them, so they can just snap and grab.

How we built it

We used javascript for the user experience, allowing users to select precise sections of the window. From here, we set up an API endpoint to process this image and grab its text. From here, we delved into prompt generation and distilling our LLM to structure the responses into Google Calendar specific data. We leveraged the llamas3 model in order to detect events and leveraged the Google API to perform the Calendar event generation.

Challenges we ran into

We had a lot of moving parts in our framework. We had a chrome extensions framework that needed to meld with image processing, our LLM, and finally API's to Google Calendar and Oauth. Marrying all these solutions took steady communication and vision from all of us. We leveraged each others skills to touch all these bases and create a product we are truly proud of.