Hive - Streamlined ML Data Collection

Build Machine Learning datasets collaboratively and easily, so you can focus on what's important.

Comment

Inspiration

Machine Learning has taken the world by storm over the past few years. It's without a doubt one of the most exciting technologies out there currently, with an almost endless number of applications.

However, despite the popularity of Machine Learning, collecting a good dataset to begin training a model can prove challenging, especially for individuals, small businesses, or those with specialized use cases. To tackle this problem we've created Hive!

Try it Out!

The project can be tried out by downloading it through the provided GitHub link. After unzipping the project, navigate to the file htnsite\main\dropbase_requests.py, and set the TOKEN variable equal to your own Dropbase API token, and within the databaseQuery function, set the REST_API_LINK, TABLE, and KEY variables equal to your corresponding values, which can be viewed from the Database information provided within Dropbase's dashboard. Next you can open your terminal, change into the directory containing the unzipped project, and run the command python manage.py runserver to start the web app at your localhost address. You will require python and django to be installed on your machine for this to work.

What It Does

Hive is a streamlined Machine Learning data collection platform. It aims to allow users to build Machine Learning datasets collaboratively and easily, so they can focus on the important parts of their project instead of having to get bogged down in trying to gather data.

Users who are seeking data for a project can make a public post on the platform with the details on what type of data they want, and what they're collecting it for. From here, other users will be able to see the post and contribute their data by uploading files, thereby fostering a collaborative environment wherein users can assist one another in assembling robust datasets for use in machine learning projects of all shapes and sizes.

More specifically, when files are uploaded by a user, they are preprocessed according to their data type, stored in a temporary CSV file, and then added onto a database within a Dropbase project using the Dropbase API.

How It Works

Hive is a web app built using Django and Firebase, with the Tailwind CSS library used for front end styling. File uploading, preprocessing, and all related functions are handled through Python. All data is pushed to and retrieved from a Dropbase database, which is interfaced with through its API, using the Python requests module.

Challenges Faced

Unfortunately, the day the Hackathon began, one of our team members started revealing symptoms of COVID-19, and as a result they were not able to fully contribute to the project, reducing the amount of manpower available to help develop our project.

Additionally, due to technical/software limitations the recorded demo had to be conducted by a single team member from their computer.

Share this project:

Updates