BirdScribe AI

Streamlit App with Image - Input
Streamlit App with Output
Detailed Description of Output
Query regarding *White Breasted Nuthatch* bird to LLM

Inspiration

Passion for Wildlife Conservation – Inspired by the urgent need to protect bird species from habitat loss, climate change, and declining populations.
Bridging AI & Nature – Leveraging Computer Vision and LLMs to make bird identification and query accessible, fast, and intelligent.
Empowering Bird Enthusiasts & Researchers – Creating a free, AI-driven tool for birdwatchers, students, and conservationists to identify and learn about birds effortlessly.
Fostering Environmental Awareness – Encouraging people to explore, appreciate, and protect biodiversity through technology-driven education.

BirdScribe AI merges innovation with impact, promoting wildlife conservation and awareness while aligning with UN SDGs 4 (Quality Education), 13 (Climate Action), and 15 (Life on Land).

What It Does

BirdScribe AI leverages resources to simplify bird identification and enhance conservation efforts;

Identifies Birds Instantly – Uses AI to detect bird species from images and videos.
Provides Expert Insights – Displays details like scientific name, habitat, size, and lifespan.
Supports Conservation – Connects birdwatchers and researchers to protect bird species.

How We Built It

Dataset Selection & Annotation – We sourced the CUB-2011 dataset consisting of 11,788 images of 200 subcategories from Caltech Vision Lab
Model Training – Trained custom state-of-the-art YOLOv11x (Large Model) on the dataset for high detection accuracy.
Inference Pipeline – Built an optimized model inference pipeline for real-time detection in images and videos.
LLM Integration – Connected Mistral-7B via Hugging Face API for detailed bird information retrieval, using prompt engineering for accuracy.
Streamlit UI Development – Designed an interactive web app consisting of AI based detection and chatbot functionality.
Deployment – Hosted the app on Hugging Face Spaces, making it publicly accessible.

Challenges We Faced

There were many obstacles and challenges that we ran into during creation of the Birdscribe-AI. The few of them were:-

Identifying the Right LLM – Finding an accurate LLM (Mistral-7B) was challenging, so we tested various models, applied prompt engineering for fast and precise retrieval of birds information.
Training custom model with limited computation resources – Trained custom model (Yolov11 ) efficiently, maximizing accuracy with minimal resources
Seamless UI – Ensuring real-time AI predictions without UI lag required a streamlined approach, so we built an Streamlit App for easy usability and enhance user interaction

Accomplishments We’re Proud Of

Real-World Impact – Developed an Computer Vision and LLMs integrated solution addressing UN SDGs, promoting sustainability.
Advanced Bird Detection – Trained a custom YOLOv11x model on CUB-2011 for high-accuracy species identification with limited resources.
Promoting Conservation Awareness – Built an AI based tool that encourages from birdwatchers to researchers about conservation and sustainability of the natural beauty of the birds at your fingertips.

What We Learned

We gained invaluable insights, especially in real-life problem-solving for bird detection, along with communication skills for documentation, team coordination, and user-centric design. Additionally, we explored project management and scalability while deepening our expertise in cutting-edge technologies like prompt engineering, model optimization, inference, LLM integration, and deployment.

What’s Next for BirdScribe AI?

Enhanced Detection – Expand the dataset and integrate more state-of-the-art (SOTA) computer vision models to improve accuracy and include new bird species.
Fine-Tune LLM for Birds – Fine-Tune LLM for Ornithology on bird-specific data for accurate, insightful responses from chatbot.
Integration with Bird Databases – Connect with platforms like eBird or iNaturalist for scientifically validated data.
Interactive Visualization – Incorporate maps to track bird sightings, migration patterns, and regional species distribution for a more immersive experience.