Google Form Filler (Gff) Tool

The Google Form Filler (Gff) is a smart assistant that knows exactly what to put in each field of the form, thanks to Google Gemini AI and learning from the documents you provide.

Comment

Inspiration

Think of all those registration or onboarding forms where you have to enter the same information over and over again. Thought: why not use AI tool to do the heavy lifting? That's where Gff comes in. If you already have all of the information in the documents, why not let an AI tool do the auto-completion for you?

What it does

Gff automates the process of filling Google Forms by extracting context from documents using embeddings and Google Gemini AI. It fills out forms based on this context, and can request additional context for questions that require more specific answers.

How we built it

Choosing Python and the following libraries helped us develop Gff quickly and keep it easy to use.

  • Google Generative AI: For smart form filling based on document context.
  • EmbedChain: To handle document contexts with embeddings and store to vector database.
  • Selenium WebDriver: For fetching full HTML text of forms
  • Rich: Made our CLI look good and interactive.

Challenges we ran into

  1. Understanding how Google Forms works: One of the first things we had to do was a deep understanding of the internal structure of Google Forms, especially the different field types. We spent a lot of time researching and experimenting with different forms to ensure that our tool could accurately identify and interact with a wide range of field types.
  2. Creating accurate prompts: Another challenge was to create precise prompts for LLM to get back exactly the information we needed. This required a balance between being specific enough to get useful answers and being flexible enough to handle a variety of form content.
  3. Small Context Window: Analyzing large forms with complex structures was particularly challenging due to the small context window limitations of early AI models. However, the introduction of Gemini 1.5 Pro was a game changer for us. Its improved ability to understand and process larger contexts without losing consistency improved our tool's efficiency and accuracy in handling large forms.

Accomplishments that we're proud of

  1. Integration with Google Gemini AI:Successfully integrating cutting-edge AI technology (Google Gemini AI) into our tool is a major achievement.
  2. Support for multiple field types: We have designed Gff to support a wide range of Google Form field types, including short text, paragraphs, multiple choice, checkboxes, and more.
  3. Interactive prompts for user: Implementing interactive prompts to guide users and provide additional context when needed. We appreciate the balance between automation and human oversight.

What we learned

One of the most important lessons we learned is the importance of iteration. Our first attempts at creating prompts for the AI were not always successful. However, through continuous trying, we improved our prompt engineering techniques.

What's next for Gff

  1. Video and image analysis: Integrate Gemini Pro Vision to analyze video and image questions.
  2. Support for multi-page forms: Improve the acceptance and processing of multi-page forms.
  3. Enable File Uploads: Add the ability to upload files for forms that require files submission.
  4. Develop a browser extension: To make Gff more accessible for users

Built With

Share this project:

Updates