Inspiration
I got inspired when i got to know about the state of farmers in India and how they are often not able to get the worth of their crops due to lack of information even though the govermnet provides the data extensively.
What it does
It lets the farmers use the AI in their native language to make use of data to get insights to make a decison on when to sell and where to sell.
How we built it
We used Databricks' ETL pipelines to load the data from AI kosh and used Spark and Databricks Jobs to schedule regular data update to keep track of prices daily. We used Spark Notebook on Databricks to train our Regression Model for prediction of future prices and then we used Databricks AI Gateway to expose an LLM endpoint for decision making and structuring. We used SQL Warehouses and Volumes and Data Lakes and we used catalogue to keep track of schemas and we used Genie for constant help and guidance. Finally we used Databricks App to host our react and fastapi app where our agentic AI rested facing the end customer.
Challenges we ran into
Data cleaning was extremeley tough as the data was very unfiltered and varied across the columns and rows and filled with null and corrupted values. We also struggled with Model training on Free Compute power so we reduced the size of our training data by splitting our dataset vertically and training on one crop only.
Accomplishments that we're proud of
All of them.
What we learned
How useful and time saving Databricks is along with how many data sources gov of india provides.
What's next for Mandi Mitra
Adding more data predictions, taking weather and distance into account while suggesting etc.
Log in or sign up for Devpost to join the conversation.