CampBuddy

• 

  Default Landing Page Light Mode

• 

  Default Landing Page Dark Mode

• 

  Sample Prompt and Response

Inspiration

We were inspired to provide a solution to a real problem that real people face in a real time constraint. Many of us use AI today but take for granted our internet access that allow us to use these powerful tools. Some individuals in remote locations like campers or hikers need the assistance of these tools but do not have the internet access that we do.

What it does

This app provides a local AI assistant to users without wifi such as hikers and campers. Users can ask survival, first-aid, and gear questions and get answers grounded in locally stored manuals, with the source document and page shown so they can verify the advice, and with a clear label whenever an answer comes from the model's general knowledge instead.

Additionally, a computer vision model is also run locally aiding users with visual tasks such as identifying different species of plants. Users can then ask follow up questions about the images they've uploaded.

How we built it

We built this app using a React-based frontend (Vite and Tailwind CSS) and a FastAPI backend that passes each user question through a LangChain RAG pipeline, which retrieves the most relevant passages from a ChromaDB vector database of survival documents and feeds them to Llama 3.2 3B running locally on Ollama. This way, inference and document search stay on-device, and each user's chat history is saved privately in their own browser.

Challenges we ran into

One of the struggles we initially faced was determining which AI model to use for our local LLM. We had to prioritize capability while still taking into account RAM usage and limits that are most common in commercial laptops.

We faced a similar challenge when choosing our computer vision model. We faced roadblocks where potential computer vision models that we wanted to use would completely misidentify plants and had to adapt to change of plans in a short notice.

We also struggled with initially setting up our model and integrating it into our backend API routes. We also struggled with integrating the computer vision element for scanning plant types.

Accomplishments that we're proud of

We are particularly proud of the RAG integration and offline Computer Vision capability of CampBuddy.

What we learned

We learned LLMs can be run locally on laptops and how Retrieval Authenticated Generation (RAG) works to improve the local LLMs accuracy. Additionally we learned more about how computer vision works, what the potential downfalls are with certain models and how using models specifically trained for a specific identifications work better than models trained for general identification.

Furthermore, we learned how starting a project with a clear design vision can actually turn general design concepts into concrete architecture and to use AI-tools to help maintain rapid development pace.

We ultimately learned how to take our development expertise and not only integrate them into an effective team setting, but use them to guide AI tools and actively control them rather than simply relying and being controlled by such tools.

What's next for CampBuddy

CampBuddy's future lies in our vision to enhance our dedication to making a local, wifi-independent assistant that can offer robust camping/survival advice. Our next endeavors involve enhancing our model's supporting resources in breadth, depth, and reliability and enhancing the user experience with features like modifying chat history and accepting voice input.

Built With

• chromadb
• fastapi
• python
• react
• vite