Inspiration
After reading the prompt, we immediately thought that we wanted to learn how to create a predictive model using Databrick's support agent and interface that completes this objective. That’s where Crop-ital Gains began.
What it does & How we built it
We used a temporal CNN to develop the prediction model, which will predict the yield amount based on the current weather patterns. To do this, we created a fixed time window of 180 days from the temperature max, temperature min, and precipitation for the model input. We preprocessed our data by putting it in the format of temperature min/max, precipitation per year, corresponding to the yield of that year. We developed a temporal block which is our main building block, containing one-dimensional convolutions in addition to dropout regularization and and relu activiation. Our yieldTCN is a sequence of temporal blocks for short-term, medium, and long-term patterns, which helps it to automatically learn different features from our raw weather data. We split our dataset into train, test, and val datasets and used the train and val datasets to train our hyperparameters. We trained our model using supervised training and used Adam and MSE loss to optimize our model. We evaluated our accuracy on the test dataset and received an MSE of 0.0105. With out newly trained model, we created AI functions in the SQL editor that allow us to ask the model in a natural language about yield based on weather data.
Challenges we ran into
We came into this hackathon without prior machine learning experience and no familiarity with Databricks. We spent nearly the first hour just navigating the UI and understanding the workflow before writing any code.
Other challenges included:
Understanding how to properly structure time-series data
Understanding what type of model to use for prediction given the Databricks model system
Databricks Notebooks kept losing our local progress when we closed our computer, causing model training to be a painful process
Accomplishments that we're proud of
We created a regression network as people who were unfamiliar with making machine learning models from scratch. Additionally, we practiced using Databricks' complex interface, and have gained a new understanding of how things worked.
What we learned
We learned how to preprocess data into a temporal format, how to create a temporal CNN, and we learned about data governance. We also learned the patience of how to build a model.
What's next for Crop-ital Gains
In the future, we hope to use the model to create dashboard that can predict hypothetical situations and show yield results as well as a yearly harvest plan for optimal growth.
Log in or sign up for Devpost to join the conversation.