-
Input the pdf, provide prompt, and add red boxes.
-
Pdf is generated and filled out.
Inspiration The inspiration behind DocuFiller AI stemmed from our previous experiences with transcription projects. Both Andrew and I noticed a recurring need among users, particularly in the medical field, to utilize transcriptions for filling out various forms. Doctors and dentists, for instance, often have to manually input information such as allergies from conversations with patients, which can be tedious and error-prone. This realization drove us to create an AI solution that could seamlessly transform natural conversations into structured data, thereby automating the form-filling process.
What it Does DocuFiller AI allows users to automatically fill out PDF forms using transcriptions of natural conversations. By providing a PDF and marking the desired fields, our AI can accurately populate these fields based on the conversation transcript. This tool simplifies the data entry process for medical professionals and other users, reducing the need for manual input and minimizing errors.
How We Built It We leveraged OpenAI's image recognition software to develop DocuFiller AI. Initially, we experimented with various techniques, including the use of random placeholders on PDFs. The idea was to identify what content should replace these placeholders using image recognition and understanding. However, we soon realized that existing AI solutions couldn't pinpoint the exact pixel coordinates of form fields. Consequently, we designed a system where users specify the coordinates, allowing our AI to accurately fill in the required data.
Challenges We Ran Into One significant challenge was the absence of an AI capable of identifying and providing pixel coordinates for input form fields. This limitation required users to manually specify the coordinates, which added complexity to the user experience. Additionally, determining the appropriate data for each form field posed another challenge, necessitating the use of OpenAI for conceptual understanding. We also faced difficulties ensuring the AI's accuracy in recognizing and processing different form layouts and content types.
Accomplishments That We're Proud Of We are proud to have created the world's first AI PDF autofill solution that does not require custom form fields with special metadata. Our tool is highly generic, adaptable to various forms without the need for predefined templates. This achievement showcases our ability to innovate and overcome the practical limitations of current image AI technologies.
What We Learned Throughout the development of DocuFiller AI, we learned about the practical limitations of image AI. While these technologies can grasp the conceptual content of images, they lack the precision required for specific measurements and coordinates. This insight highlighted the need for customized solutions to bridge the gap between conceptual understanding and practical application.
What's Next for DocuFiller AI Looking ahead, we aim to expand the capabilities of DocuFiller AI. Our goal is to eliminate the need for users to manually draw boxes on PDFs by training a model to recognize existing form fields within the PDF itself. This enhancement will make the tool even more user-friendly and efficient. Additionally, we plan to support a broader range of use cases, further refining our AI to better serve diverse industries and applications.
Log in or sign up for Devpost to join the conversation.