I am very interested in deep and machine learning. I play around with pytorch, keras, and scikit learn often. However, finding and acquiring the specific data that I need is often hard. The data available on the internet is too general or does not suit my purposes. I created crowdata to solve this issue.

What it does

Crowdata is a platform where users can request data and other people who have access to that data can submit it to earn monetary rewards. For example, say you need microscopic images of a cell when it was treated with a certain chemical to train a convolutional neural network. However, you don't work in a lab and specific images of these cells aren't available online. So, you go to crowdata and make a listing for the specific cell images you need. Then, another crowdata user who happens to work at a lab with access to these cells sees your listing. This user goes to their labs, takes the images you requested, and submits them to you through the crowdata platform. They are paid the price that you set, and you receive the images you need to train your deep learning model. This way, getting specific images for machine or deep learning is no longer an issue.

How we built

Crowdata was built using the MERN stack. MongoDB was used to store user info, listings for data, and the data itself in the form of base 64 images. Express was used to send http requests to the MongoDB database. POST requests were used to add new users, update users' balances, add new listings, and add data to the listings asking for it. GET requests were used to retrieve listings to display to the user. React JS as well as react-bootstrap was used to make the front end user interface where people could make listings and answer other peoples' by supplying data. Google OAuth was used for the login. Finally, node.js was runtime of the backend. To try it out for yourself, create a mongoDB Atlas database, get OAuth credentials, and visit the github repository

Challenges we ran into

Figuring out how to store the images was a big challenge. This was the first time I really worked with mongoDB, so it took me a while to decide to store images in base 64. Additionally, setting up the routes for the database using Express was more complicated than I expected, so I had to spend a lot of time on that.

Accomplishments that we're proud of

I am very proud of putting together a full working website. Actually finishing the project in the allotted time was difficult, but it was well worth it.

What we learned

I learned how to make a proper full stack web app using Javascript.

What's next for crowdata

Crowdata can start integrating actual payments and percentages to make a revenue and become an actual business. Also, allowing other types of data to be requested on the platform like videos and audio is the next step. This way, crowdata can expand to more users and make machine and deep learning easier for all.

Share this project: