RESILITREE

Team RESILITREE

Inspiration

Our inspiration for RESILITREE originally came from the idea of tackling hurricanes, the most prevalent natural disaster in Florida by far. Being from Florida all my life, I know all too well how dangerous treefall could be. The most dangerous treefall story that I heard from someone in my life was in high school. My chemistry teacher was involved in an accident where a big tree snapped in half in front of her house, caving in her roof during the storm. Although her family came out of that accident with minimal injuries, that will not always be the case. Hence the creation of RESILITREE . We wanted RESILITREE to be an educative tool keeping people informed with the information they need when it matters most.

What it does

RESILITREE is an innovative application that helps users assess the risk of treefall during natural disasters like hurricanes, while also offering real-time disaster preparedness advice. The app features a Tree Fall Risk Prediction tool that allows users to predict whether the a tree is fall-prone using an image of the tree. Additionally, based on the identified tree species, the app explains why this species of tree is prone or not prone to treefall as well as suggests actionable precautionary measures to help stabilize trees and safeguard homes. To complement the risk prediction, RESILITREE includes an interactive Hurricane Relief Chatbot meant to provide users with context-specific guidance on disaster preparedness, offering insights on what to do before, during, and after a natural disaster. The chatbot engages users through a continuous conversation format, responding to queries and offering safety tips tailored to the user's specific needs. Whether it’s predicting risks, providing real-time safety advice, or simply answering users' questions, RESILITREE empowers communities to take proactive measures in disaster scenarios.

How we built it

We built RESILITREE using a combination of computer vision and generative AI. Our project leverages deep learning models trained using PyTorch to assess the risk of treefall during hurricanes. We incorporated a Convolutional Neural Network (CNN) to analyze uploaded tree images and predict their stability with high confidence. To generate text for both the Fall Risk Prediction and Hurricane Relief Chatbot we integrated IBM’s Granite-13B model. For the UI, we used Streamlit, which enabled us to create an app focused on giving a user-friendly experience with clear visual indicators and explanations. To keep sensitive information secure, we adopted best practices for managing secrets, including the use of environment variables and Streamlit's secrets management. Our codebase was structured to facilitate easy future expansion.

Challenges we ran into

Some challenges that we ran into for this project was model training. The first challenge came from acquiring our data. There are not an abundance of common Florida tree datasets believe it or not which is why we had to manually download and annotate the data ourselves. We made sure to only pick data that was free use. This also involved having to research specific tree species that either excelled or crumpled during hurricanes. Since we were limited on time we decided that having 3 species of both categories would be enough for now in order to train our model. The second challenge came from experimenting with different deep learning models. We knew from the start that we wanted to leverage the powerful capabilities of an already pretrained model due to the limits of our dataset as well as the time constraint. Leveraging transfer learning techniques, we knew that it was possible to train these models to fit our specific needs. The problem came from model selection. We experimented with a couple different deep learning models and came to find that EfficientNet brought about the best results. The third challenge came from building the Hurricane Relief chatbot. Initially, the chatbot did not give us the output we were looking for, but after leveraging WatsonX.ai provided by IBM, we were able to easily experiment with different prompting techniques. This allowed us to configure our prompts to give us a more clean looking output with the correct information about hurricane safety.

Accomplishments that we're proud of

We are proud of the fact that our application serves a more niche aspect of hurricane safety. It might surprise you that many Floridians don’t consider securing theirs trees near their houses before a hurricane. It is our hope that RESILITREE would be able to provide homeowners with detailed instructions about treefall prevention as well as any other general hurricane safety tips they may need. We are also proud of the fact that the scope of our idea combines our backgrounds in AI as well as education. Our friend Tinki, took a big leap for this hackathon being from an math education background. We appreciate his sincerity for wanting to learn frontend python for the sake of this project. Lastly, we are proud of the fact that Anany and I were able to utilize our technical expertise in computer vision. Being apart of the first ever incoming class for UF’s new graduate program AI Systems, we hope to represent our program in a positive light this hackathon.

What we learned

What we’ve learned from this hackathon is how important communication is, especially when it comes to working with people from different disciplines. Our team was lucky to be made up of individuals who understood that. Being able to communicate our different skillsets and what we could bring to the table was what ultimately led us to RESILITREE. During the initial planning stage of the project was where this was the most prevalent. Our thought process behind RESILITREE started with identifying niche problems that could be solved using computer vision and generative AI. The idea for our project came to fruition after we all shared our collective experiences with natural disasters and how these experiences could be used to combat them in the future.

What's next for RESILITREE

Some ideas that we thought of adding in the future for RESILITREE was to implement a camera functionality as well as migrate the software onto mobile devices. That way users are able to more easily bring RESILITREE with them on the go. This would eliminate the need to upload an image onto RESILITREE and would instead provide the fall risk prediction by simply snapping a photo. Users would also be able to utilize the more personalized chatbot whenever they please once the software is migrated to the phone as well. Another idea we had for RESILITREE was to change the UI to be more aesthetically pleasing. This will idea will be important in order to make the software compatible on mobile devices since Streamlit is not meant to be displayed on a much smaller screens to begin with. As for the computer vision model itself, we thought about creating a much larger dataset in the future to make the model more robust towards predicting fall risk for all trees in Florida.