Compiler Assistant

No time to search and digest evolving AI knowledge? Compiler Assistant generates a compiled article to answer your AI tools questions through conversations. Learning AI gets easier and more efficient.

Comment

Inspiration

Generative AI tools grew explosively in 2023 and continue to rise rapidly in 2024. It’s undoubtedly a burden for an individual who wants to grasp new knowledge because it requires a significant amount of time and effort. Therefore, we designed an innovative Agent, Compiler Assistant. It's operated by natural language prompts to give users a one-page compiled, practical article based on user requirements through simple conversations.

What it does

Compiler Assistant can search and integrate information from the internet, output a compiled and practical article for your AI tools related questions. What's more, users do not need to worry about the reliability of the generated information, nor do they need to worry about the accuracy of their query terms. Compiler Assistant has a query rewriting feature that helps users fulfill more precise terms for searching. Compared to traditional self-searching and learning, Compiler Assistant saves 70% of the time spent on searching and digesting contents, significantly lowering the threshold for learning new AI tools.

How we built it

We maintained our project using a full cloud (GCP) environment. The whole process can be broken into the following parts:

  • Front-end: Vertex AI Agent Builder
  • Backend: Python Flask
  • Deployment: Cloud Run
  • Language Model Service: Gemini 1.5 Pro
  • Article Storage: Cloud Storage

Challenges we ran into

  • Prompt Design
    • Challenge: It’s crucial to design prompts that can guide LLM to generate high-quality, relevant content. The diverse expressions and needs of different users increased the difficulty of designing suitable prompts.
    • Solution: We applied few-shot learning and chaining in the prompt examples. We also conducted multiple test cases and feedback iterations, eventually formed a standardized prompt template. These templates can flexibly respond to different user needs.
  • Query Rewriting
    • Challenge: Users often are unsure how to express their needs, leading to inaccurate or irrelevant initial search results.
    • Solution: We developed an intelligent query rewriting system that uses contextual understanding and semantic analysis techniques to automatically identify keywords and intent in user queries. This system also provides query suggestions to help users better express their needs.
  • Cost Efficiency and Service Stability
    • Challenge: Ensure service quality while controlling operational costs and maintaining high availability.
    • Solution: We used GCP Cloud Run as the deployment tool which charges by requests to reduce costs. Additionally, it features auto-scaling mechanisms to maintain overall high availability.

Accomplishments that we're proud of

  • Efficient Prompt Design
    • Achievement: Successfully designed a set of flexible and efficient prompt templates that can adapt to different user needs and guide the AI to generate high-quality content.
    • Impact: This not only improved user satisfaction but also significantly reduced the time and effort users spent on repeatedly adjusting prompts.
  • Intelligent Query Rewriting System
    • Achievement: Developed an intelligent query rewriting system that can automatically identify user intent and provide more accurate query suggestions.
    • Impact: This greatly increased the relevance and accuracy of search results, allowing users to obtain useful information even when they are uncertain about their needs.
  • Cost Efficiency and High Stability
    • Achievement: Successfully reduced operational costs while improving service stability through dynamic allocation of cloud resources and multi-layered service monitoring.
    • Impact: This enabled us to provide high availability and high-quality services to users while controlling costs, further enhancing our competitive advantage.
  • Significant Improvement in Output Quality
    • Achievement: Compared to the initial version, we significantly improved the accuracy and quality of the output after adjustments.
    • Impact: This not only allowed users to quickly obtain the information they need but also lowered the threshold for learning new AI tools, saving users a lot of time and effort.
  • Completing the Project on Time
    • Achievement: Despite starting late due to other commitments, we completed the project on time and added most of the envisioned features planned during the ideation phase.
    • Impact: This is a proud achievement, demonstrating our ability to complete the project under tight schedules and deliver a complete product.
  • Excellent Team Collaboration
    • Achievement: We made a great team!
    • Impact: The collaborative efforts of the team significantly contributed to the success of the project.

What we learned

  • Gemini 1.5 Pro
  • Vertex AI Agent Builder
  • Grounding with Google Search
  • Function calling through OpenAPI schema. We did the conversion using Swagger editor.
  • GCP tools

What's next for Compiler Assistant

We'd like to have more components for Compiler Assistant, including:

  • Keep chat history with a great memory.
  • Enable multimodal input and output.
  • Build multi-agent with additional sources such as Medium.com and LinkedIn post. This helps get comprehensive results in terms of the latest AI tools developments.
  • Build reasoning engine using LangChain.
  • Integrate with Messenger App to connect with more users.
  • Allow users to edit markdown output files.
  • Develop a document management feature for searching with multiple Gen AI tools such as Gemini and ChatGPT.

Built With

Share this project:

Updates