TideCast

Inspiration

When we looked at this hackathon’s beach theme, we wanted to build something that truly useful for anglers. but we realized a huge problem right away. No one wants to mess with a slimy measuring tape on the sand, and dragging out the process is dangerous for the fish. We realized the solution was literally in our hands. Everyone holding a fish has their hand in the photo. If we could use the angler's knuckles as a virtual ruler, we could solve one big problem they face.

What it does

TideCast runs entirely on one page with dedicated tabs for more information. We designed it as a smooth, 3D scroll driven journey that flows through four main features:

Explore the Waters: A 3D globe that pulls in real OBIS fish records and Global Fishing Watch data so you can see exactly where to fish near you.
Log a Catch: You upload a photo of your catch with your knuckles visible. Our model instantly identifies the species from trained fish identifying model, traces the fish from snout to tail, maps your knuckles, and calculates length, weight and other really cool interesting stats about the fish accurately.
The Leaderboard: Catching rare or large fish earns you points based on real marine biology metrics, instantly ranking you against other locals.
Species Guide: It has a built in field guide for the 30 species TideCast currently tracks.

How we built it

We wanted to keep the stack lightweight so it could run instantly without a massive setup. The entire project is served by a single Python server (server.py) that is connected to several databases and training files. On the frontend, we built a custom HTML/CSS/JS design ("Backbone"). For the computer vision, we used the Gemini API. When you upload a photo, the model processes it on a 1000x1000 grid, returning the species name, six coordinate points along the fish's curve and the span of your knuckles (from index to pinky). The real magic happens in our math engine (measure.py). We use average human knuckle spans (or the user's custom measurements) as a physical ruler to translate pixel distance into centimeters. Once we have the true length, we run it through the scientific length-weight formula (W = a * L^b ), replacing the exact a and b constants for that specific fish species from our local database. Also, in the explore page, we had implemented many unique features. For example, we had implemented a feature that gets vessel information. What we mean is that in a certain area, we could get information regarding whether or not a certain vessel was found in that zone at a point in time. This could help a fisherman determine the value of the water, and whether or not they can properly get fish in that area. Also, we give information regarding who can use specific oceanic zones(whether it's government protected, commercial usage, private, etc.)

Challenges we ran into

Calculating weight was our hardest technical challenge. AI can't accurately guess weight from 2D pixels because it can't see volume. We implemented the marine biology formula (W = a * L^b ), but because b is an exponent, any slight pixel inaccuracy in our knuckle measuring would cause deviations in the final weight. Our evolution in regards to evaluating the weight of the fish were through three key stages. Our first version asked for the AI to eyeball the length, which cause major problems as there was no real way to validate our results. Our second version had used a reference point, being the knuckles of the person holding the fish. When we find the length of the knuckles, we can divide by the amount of pixels the knuckles take to get the length per pixel. Using this conversion factor, we hoped we could accurately figure out the length of the fish. However, since we had no realistic way to test this method(since we couldn't figure out the length of a random person's knuckles), we had to pivot into our final version. In this version, we had implemented a free learned model called the fish length predictor, which embeds the image with DINOv2 , scales the features of the image, and runs the picture through a trained AutoML/XGBoost regressor. There it returns the length in centimeters. Through this final method, we were able to get results that were much more accurate in weight than before. Also, two fish of the same length can weigh different amounts in the wild. We had to write a custom mathematical range function. Our server now calculates an range that accounts for both the potential pixel distance error and the natural plumpness spread for that specific species to make sure our leaderboard stays accurate and realistic.

Accomplishments that we're proud of

We set a massive goal to build four distinct features into one unified web app, and we are super proud that we fully implemented them. From the live OBIS data mapping to the knuckle measuring math, everything works together. While building, we had to solve a bunch of technical bugs, especially around making sure the draggable frontend UI perfectly matched the backend Python calculations.

What we learned

This hackathon taught us a lot about connecting different parts of a project and making everything work together. We learned how to connect our frontend to a Python backend and keep the draggable points synced with the calculations. We also got to work with computer vision and figure out how to get the model to identify specific coordinates on a fish instead of just making random predictions. Building the 3D website was also a new experience, especially figuring out how to make it look good without making it hard to use. Obviously, we ran into a lot of bugs along the way and fixing them close to the deadline was stressful. But we learned how important it is to stay calm, read the errors, and actually figure out what's wrong instead of randomly changing code and hoping it works.

What's next for TideCast

Hyper Accurate Weight & Volume: Allow users to save their exact hand measurements in a profile for sub millimeter precision. We also plan to train the model to estimate fish girth and 3D volume, transforming broad weight ranges into exact weights.

Database Expansion: Scale our backend from 30 species to hundreds by pulling a and b length-weight constants directly from FishBase and NOAA. We’ll also dynamically adjust these formulas based on season and region to taking spawning weight into account when calculating.

Smart Scale & Environmental Alerts: Teach the model to auto-detect other reference objects like phone cases or rod handles when hands aren't in view.