Local file identification and sorting prototype

Inspiration

We were considering, what would be generalized use of AI within our daily lives. and we came to a conclusion that not everybody is good at file management.

What it does

This app acts as a prototype for a sorting application that uses AI to Identify notable elements that would be tagged on the file. This would allow for an easier time organizing each file.

How we built it

The methodology we though of was simple, we can have flask web application be the frontend of the project, there would be a designated folder where the user can input desired files into for tagging and analyzing. When we run the analyzer, the files get sent to the Gemini API for identification and tagging. this would result into a caching of filename followed by a list of tags. from there we rearrange the files into a key pair (tags) and value (file's name), which enables us to use a map reduce technique to group all the files with the given key pairs. This would all then gets returned onto the front-end where we can see each tag and their associated files listed below.

Challenges we ran into

The problems we ran into during development mostly related to rate limits and response wait time. Since our prototype sends a request to Gemini's API sequentially for each file, we lose time waiting for an API call to finish. One workaround would be to use a thread pool executor in Python. We wouldn't necessarily have true multithreading because of Pythons GIL but we can still reduce overall runtime by making API calls while we wait for prior ones to respond. As for rate limits, we propose batching files with each call. This would help with the requests per minute but at the expense of token usage and response time.

Accomplishments that we're proud of

During development we ran into unreliability with extracting text from PDFs. We pivoted to converting the PDFs into images before passing them onto the Gemini API for processing.
Implementing a Map-Reduce inspired pipeline

What we learned

As a team, we learned a lot about rapid development and integrating Gemini API into a web application. We also gained experience with API constraints and the tradeoffs that can come with optimizing speed and quotas.