Ornipedia

Inspiration

One of our group members enjoys going "birding" as a hobby. Upon discovering that one of the tracks for this hackathon is Machine Learning a few ideas were presented, but ultimately we went with something that was of personal interest.

What it does

This application will access a user camera (with permission) and detect a bird through it. Upon detection, a TensorFlow model is used to identify the bird species. Given this information, an LLM(llama) will provide the user with educational information, and various educational resources to further their learning.

How we built it

We built it using React-Native for the front end, TensorFlow for Machine Learning, an LLM(llama) for the content generation, and Django for the backend.

Challenges we ran into

We encountered an error attempting to convert a TensorFlow model with type keras to type tfjs(TensorFlow JavaScript).

Accomplishments that we're proud of

We resolved our challenge by deciding to implement a Django backend instead, to facilitate the classification request with the Python model.

What we learned

We learned that the compatibility between technologies is not always the smoothest.

What's next for Ornipedia

A future feature could include recording the encyclopedia entires as they are generated in a database, as well as the ability for users to create an account so they may view all of their discoveries.