BirdScribe AI

Streamlit App with Image - Input
Streamlit App with Output
Detailed Description of Output
Query regarding *American Three Toed Woodpecker* bird to LLM

Inspiration

Commitment to Wildlife Conservation – Driven by the urgent need to protect birds from habitat destruction, climate change, and population decline.
Merging AI with Nature – Utilizing Computer Vision and LLMs to enable fast, intelligent, and accessible bird identification.
Empowering Enthusiasts & Researchers – Providing a free, AI-powered tool for birdwatchers, students, and conservationists to identify and learn about birds effortlessly.
Promoting Environmental Awareness – Encouraging biodiversity appreciation and protection through technology-driven education.

BirdScribe AI integrates innovation with impact, aligning with UN SDGs 4 (Quality Education), 13 (Climate Action), and 15 (Life on Land) to support global conservation efforts.

What It Does

BirdScribe AI simplifies bird identification and enhances conservation through:

Instant Bird Recognition – AI-powered detection of bird species from images and videos.
Comprehensive Bird Insights – Provides key details such as scientific name, habitat, size, and lifespan.
Inspiring Conservation Action – Raises awareness about bird species and their habitats, encouraging people to appreciate, protect, and contribute to avian conservation efforts.

How We Built It

Dataset Selection & Annotation – Collected and labeled the CUB-2011 dataset, featuring 11,788 images of 200 bird species from Caltech Vision Lab.
Model Training – Developed a custom YOLOv11x (Large Model) for high-accuracy bird detection.
Inference Pipeline – Built an optimized pipeline for real-time image and video analysis.
LLM Integration – Integrated Mistral-7B via Hugging Face API, using prompt engineering for accurate bird information retrieval.
Streamlit UI Development – Designed an intuitive web application featuring AI-based detection and chatbot functionalities.
Deployment – Hosted the app on Hugging Face Spaces, ensuring public accessibility.

Challenges We Faced

While developing BirdScribe AI, we encountered several challenges:

Choosing the Right LLM – Finding an accurate model like Mistral-7B required extensive testing and prompt engineering for fast, precise bird information retrieval.
Training on Limited Computational Resources – Optimized training of the YOLOv11x model to maximize accuracy with minimal hardware.
Building a Seamless UI – Ensured real-time AI predictions without lag by developing a lightweight Streamlit application for an enhanced user experience.

Accomplishments We’re Proud Of

Real-World Impact – Created an AI-driven tool that aligns with UN SDGs and promotes sustainability.
High-Precision Bird Detection – Successfully trained a custom YOLOv11x model on CUB-2011 for accurate species identification despite limited resources.
Conservation Awareness – Developed an accessible AI tool that empowers birdwatchers and researchers to contribute to avian conservation efforts.

What We Learned

This project provided invaluable experience, especially in real-life problem-solving for bird detection, along with communication skills for documentation, team coordination, and user-centric design. Additionally, we explored project management and scalability while deepening our expertise in cutting-edge technologies like prompt engineering, model optimization, inference, LLM integration, and deployment.

What’s Next for BirdScribe AI?

Improved Detection Capabilities – Expand the dataset and integrate state-of-the-art (SOTA) models for enhanced accuracy and species coverage.
Fine-Tuning LLM for Ornithology – Train an LLM on bird-specific datasets to improve chatbot accuracy and insightfulness.
Integration with Bird Databases – Connect with platforms like eBird and iNaturalist to provide scientifically validated data.
Interactive Visualization – Implement maps for tracking bird sightings, migration patterns, and regional distributions to enrich the user experience.