Inspiration
I wanted to know what rocks my dog brought from the park whenever we went from walks.
What it does
Identifies gemstones, rocks or stones providing info like color, price, location and an AI chat assistant.
How we built it
It's a native iOS project with backend hosted in Railway built with Flask. It receives the user image in baes64, then sends it to the proxy, parses the data, makes the instructions, sends it to OpenAI's Vision API, wait, receive the json-formatted response, send it back to the user and parse the info. Same applies for the chat.
Challenges we ran into
- Cropping the image to save-up space and get better results.
- Make a good instruction that always parses results in json format to be decoded properly by front.
Accomplishments that we're proud of
- Fast gemstone identification response (~4 seconds).
- Results have a confidence level so the user can change them if the model didn't interpret the image - correctly
- If the identified gemstone results is updated manually the model takes the context of the most accurate gemstone result.
Log in or sign up for Devpost to join the conversation.