Inspiration
We interact with plants almost everyday -- breathe in their oxygen, enjoy their vegetables, and are comforted by their presence.
We rarely take the time, though, to learn about their history, origins, and uses. With FloraQuest, you can collect flora virtually and learn at the same time! Like Pokemon Go but for plants!
What it does
Open the camera to scan a plant or one of its leaves, and AI agents will tell you the plants uses, regions, and even if it has a disease. The newly discovered plant will be logged on the public map for all to see and learn about. You can open this map to see and learn all of the flora others have cataloged around you. Access a map to see the existing plants that have already been discovered around you, and go collect them. If you are the first to catalog a new plant, you get extra points!
How we built it
We used React-Native to build an interactive mobile map that allows users to send images into a Node.js backend that points to Gemini API combined with a Fine-tuned ResNet50 model. Fine-tuning the models was an important part of getting deployable accuracy for plant identification. Upon a successful classification of the plant, the data is first saved into a PostgreSQL database then forwarded back to the user, who gets the plant added to their garden and map.
Challenges we ran into
On the first night we had a lot of trouble running Expo to get development builds up for testing. Some of our computer's suffered from lack of hardware capability, others suffered from dependency issues, so front-end development was delayed about 13 hours. For fine-tuning the ResNet50 models, many trials were ran with intel's developer cloud with no success, but after some trial and error we were able to produce a usable model. Designing a layout for a mobile application was not within any of our personal experience, so we had to bounce a lot of feedback between one another to optimize the look and feel of FloraMap. One of our members was an exchange student from France, they suffered with working with people they didn't know.
Accomplishments that we're proud of
We finished our application within the 36 hour deadline, produced three working models, a highly functional front-end, and a killer design. Seeing the amazing result we got, we're all really satisfied.
What we learned
We learned to overcome our doubts about finishing the project in time just because there were setbacks. Putting our heads forward and pushing through the storm to achieve our goal is the biggest takeaway for us.
What's next for FloraMap
Including more gamification features and adding more agents to improve the classification accuracy as we're aiming to do an open-source project, we hope other developers contribute to FloraMap because it will have a larger impact with some time.
Log in or sign up for Devpost to join the conversation.