ShelterMEON

Implementation of vapi ai
implementation of google maps and list of shelters available close to area

Inspiration

Our inspiration for this project stemmed from the alarming rise in homelessness rates, as revealed in recent studies. Based on 2023 data, approximately 8,323 people in San Francisco alone were experiencing homelessness. Nationwide, 39.3% of the population faced homelessness—the highest rate recorded since 2007. These statistics underscored the urgent need for accessible solutions to connect homeless individuals with nearby shelters.

What it does

Our project aims to address the pressing issue of homelessness by leveraging advanced technologies to create an accessible platform for individuals seeking shelter. We utilize Vapi as the primary assistant to facilitate communication between various large language models (LLMs). This enables efficient data exchange and interaction, enhancing the user experience. To provide tailored suggestions based on user location, we integrate Gemini AI. This component analyzes the user’s input and geographic data to offer relevant recommendations for nearby shelters. We also employ Vapi for speech-to-text rendering, allowing users to communicate their needs verbally, making the platform more user-friendly. Deepgram serves as our transcription service, converting spoken words into text accurately and quickly. Additionally, we incorporate Fetch.ai to retrieve real-time data from Gemini AI. This integration allows us to generate an interactive map that updates according to the user’s location, displaying available homeless shelters nearby. By combining these technologies, our project creates a dynamic and responsive solution, helping users find and navigate to shelters efficiently. Ultimately, we strive to make a meaningful impact on the lives of those experiencing homelessness.

How we built it

We utilized the MERN stack (MongoDB, Express.js, React, and Node.js) for building the application. This choice allowed us to create a dynamic and responsive user interface while maintaining a robust back-end. We used Sequelize to interact with our MySQL database, simplifying the process of defining our models, handling associations, and performing database operations. Our models, such as User, Shelter, and Search, were designed to manage the relevant data for our application. To enhance user interaction, we integrated Vapi, which facilitates communication between various large language models (LLMs). This component allows users to provide input through voice commands, making the platform more accessible. or personalized suggestions based on user location, we utilized Gemini AI. This technology analyzes user input and geographic data to recommend nearby shelters effectively. Deepgram was employed as our transcription service, accurately converting spoken input into text. This ensures that our voice recognition feature operates smoothly and reliably. We implemented Fetch.ai to retrieve data from Gemini AI. This enables us to display an interactive map that shows real-time information about available homeless shelters based on the user's location. The combination of these technologies allows users to easily find and navigate to nearby shelters. Our platform not only presents shelter information but also provides real-time updates, improving the user experience and making it easier for individuals in need to access resources.

Challenges we ran into

We ran into several challenges throughout our project. As a team, we were relatively new to AI and faced difficulties understanding the documentation and implementing key features. For example, we struggled with expired tokens, which led us to generate an SSH key to reconnect to the local repository, and had trouble running the server and comprehending Gemini AI's structure. We also had a hard time assembling the team. Christopher didn’t find a team until 17 hours into the event, and Rafayet had his flight rescheduled, causing him to arrive late. Implementing SingleStore's features was tricky because we had trouble distinguishing it from SQL. Additionally, we were learning to work with five new LLMs, which added a significant learning curve to our project.

Accomplishments that we're proud of

We are incredibly proud of the progress we made in such a short time, especially given the challenges we faced. Despite being relatively new to AI and working with unfamiliar technologies, we successfully developed an impactful project using as many as five LLMs. Our ability to adapt, learn on the go, and overcome obstacles like team coordination and complex AI structures allowed us to create something meaningful that addresses a real-world issue. This accomplishment highlights our determination and our ability to leverage cutting-edge tools to make a difference.

What we learned

Collectively, we learned a great deal throughout the development of this project. As a team, we gained experience in working with AI technologies and LLMs, especially how they can be integrated to create real-world solutions. We learned how to navigate challenges related to implementing unfamiliar tools like SingleStore, Vapi, Gemini AI, and Deepgram, and how to manage errors like expired tokens and server issues. We also improved our understanding of how these systems interact, enhancing our problem-solving and collaboration skills.

What's next for ShelterMEON

Looking ahead, future development for this project would focus on refining our AI-driven processes, expanding the data sets for more accurate shelter location suggestions, and improving the user interface to make it more accessible. We aim to enhance the real-time interactivity of the map feature and further streamline how we integrate speech-to-text functionalities with Gemini AI for even smoother interactions. Most importantly, we see future potential in scaling this project beyond San Francisco to help individuals facing homelessness in other regions.