Critter Capture

• 

  Logo

Inspiration

Our project was originally inspired by the game Pokémon Snap. In the game, your goal is to take photographs of Pokémon, which will then get reviewed for a score. We hope to make this experience fun and engaging, to encourage people to gather data for naturalists.

What it does

Our project allows a user to upload any photograph of an animal. The program will classify the animal and give the user information about this animal. This photograph will then receive an educational summary, provided by generative AI.

Our Goals

Our goal was to develop a gamified system where users can input images of wildlife to work towards achievements and receive scores, provide these images to scientific data gathering, experiment with AI Image Classification, and build a fun game for people to play. We hope that this project will educate people about the ecosystems surrounding them, and encourage people to get outside and be interested and involved with the environment.

How we built it

We build a classifier using a combination of HTML, CSS, PEFT Transformers. We fine-tuned an AI model provided by Microsoft ResNet 18 on a dataset from iNaturalist to detect animal species. Our current website implements Claude to deliver an AI generated text rundown of the user's uploaded image.

Challenges we ran into

We had a great level of difficulty in finding a suitable AI model to use. In the end, we determined to fine tune our own. We also struggled with implementing this AI into our front-end, as all of us are primarily back-end developers.

Accomplishments that we're proud of

We fine-tuned ResNet18, which was a new experience for most of our team. We created and polished a website, which was a new experience that none of us had done before.

What we learned

We learned how to fine tune an AI model. We learned how to implement multiple interesting features, such as an interactive map, uploaded image previews, and AI generated summary text.

What's next for Critter Capture

We hope to add additional gamification to our website, and maybe build a mobile app. Depending on the popularity of our project, we will start hosting publicly instead of just on our test server. We want to work on providing gathered data to naturalists.

Built With

• claude
• css
• github
• html
• huggingface
• javascript
• llava
• peft
• python