SnowGuide

how the UI starts
Muhammad Saad - Team Leader for Project SnowGuide
Muhammad Ali - Lead Developer
Khalid Waleed - Developer

Inspiration

The inspiration for creating SnowGuide stemmed from the need to streamline the process of accessing Snowflake documentation. Developers and new users often find themselves spending considerable time searching through extensive documentation to find relevant information. By creating a Retrieval Augmented Generation (RAG) with LLMs (Large Language Model) that leverages Snowflake's documentation, we aimed to simplify this process, saving time and enhancing productivity.

What it does

SnowGuide uses Snowflake documentation to answer user queries in the most efficient way possible. It provides users with precise answers and url links to the relevant documentation, ensuring they have access to comprehensive information quickly. The reason for adding URL links is to build trust with users in RAG and LLMs application

How we built it

** 1. Gathering Documentation:** We collected all available Snowflake documentation. ** 2. Preprocessing: ** The documentation was converted to text and then chunked into manageable sections. ** 3. Creating Embeddings: ** Using the Arctic Embed model via HuggingFace, we created embeddings for the text chunks. ** 4. Storing Embeddings: ** These embeddings were stored in a vector database using FAISS (Facebook AI Similarity Search). ** 5. Similarity Search Strategy: ** We developed a strategy for similarity search to ensure the data stayed within the input token limit of the Arctic Instruct model. ** 6. Answering Questions: ** The Arctic Instruct model was utilized to generate answers to user questions based on the retrieved documentation chunks. ** 7. Deployment: ** The application was hosted on Streamlit Cloud, making it accessible to anyone.

Challenges we ran into

_ we had 2 big challenges _

One of the main challenges was managing the vast amount of documentation and ensuring the chunking process did not lose context or important information. Another challenge was optimizing the similarity search to stay within the token limit without compromising on the quality of the answers.

Accomplishments that we're proud of

We are proud of successfully creating a tool that significantly reduces the time developers and new users spend searching for information in Snowflake documentation. The integration of advanced models like Arctic Embed and Arctic Instruct, along with the effective use of FAISS, showcases our ability to leverage cutting edge technology of RAG, Vector DB and LLMs to solve real world problems.

What we learned

Throughout the development of SnowGuide, we learned the importance of efficient data preprocessing and the complexities involved in creating accurate and contextually relevant embeddings. We also gained valuable experience in optimizing search strategies and managing token limits in language models.

What's next for SnowGuide

Looking ahead, we plan to enhance SnowGuide by incorporating more advanced features such as multi modal RAG for more advanced queries and expanding the database to include more comprehensive documentation. Additionally, we aim to gather user feedback to continually refine and improve the tool, ensuring it meets the evolving needs of Snowflake users.