Sozu | Devpost

Creating a job
Completing the work
Reviewing the batch

Inspiration

Sozu is a proof-of-concept which leverages the nature of smart contracts to provide trustless, permissionless paid work to anybody in the world a la Mechanical Turk.

Amazon Mechanical Turk is a crowdsourcing marketplace which allows approved participants to complete tasks in exchange for payment. This accessible form of labour provides income for many thousands of workers, primarily in the United States, but suffers from issues of fairness, trust and transparency. Amazon must approve your participation, the platform is centralized, and workers suffer from power imbalances in the system.

Workers frequently have their work taken for free, as their work is sometimes exposed to the client and then rejected. They can also be rejected on the basis of missing "gold standard" questions, which are often engineered to be answered incorrectly at a higher rate to justify withholding payment.

What it does

This project presents a single type of task as a demonstration: Image dataset labelling. To train machine learning algorithms, up to millions of images must be accurately labelled by humans before they can be used. With Sozu, the owner of a dataset can upload their data to IPFS and create a smart contract Job endowed with Ethereum. A worker can reserve a batch of images, label the data, and submit their answers. We must avoid a few pitfalls to successfully solve the aforementioned problems:

The whole work can not be revealed to the client, or else we must trust them to pay.
The work which will reviewed cannot be known to the client, or they can stuff those tasks with the work they are actually interested in.
The work which will be reviewed cannot be known to the worker, or they can answer only those question correctly and receive payment for the entire batch.
Avoid "gold standard" questions if possible.

I utilize Chainlink's Verified Random Number Generation in what I believe is a novel method in this domain for fairly verifying the quality of submitted work without compromising it. When a worker has completed labelling the images, they commit to the Job contract a hash of the future IPFS file (determinable beforehand) containing the work and the merkle root of a tree containing their answers. Upon committing the root, Chainlink VRF randomly determines a fraction of the work to be exposed for review, which the worker then submits in plaintext along with a merkle proof that they are the authentic answers which were originally committed. Because neither party can know ahead of time which answers will be chosen, they can not cheat the system through the above methods. Further, the public nature of the blockchain offers total transparency to the nature of the work, the compensation, and the terms of review.

After a batch of answers is accepted, the worker uploads their answers to IPFS and submits the matching hash in order to withdraw payment. This basic method can be modified to support a large variety of types of work, including transcription, translation, surveys, classification, data verification and more. The ultimate goal is to create a fair, decentralized global marketplace where anybody can provide labour for anybody else.

How we built it

This project uses Solidity smart contracts, a React front-end, and IPFS for storing and transmitting data. Chainlink VRF is used to generate the randomness that selects tasks for review. It is deployed on the Kovan testnet and I used Infura's free tier for deployment and testing.

Limitations

I am new to Solidity and blockchain as of this Hackathon. I am also a solo hacker! As such, this project is not quite complete, and I was unable to fully realize it in time for submission. A more complete version of this project would include interaction with IPFS through Chainlink's oracle nodes to confirm that the contents of the submitted answers actually exist on the network, as well as an IPFS node which listens for contract events and pins the finally submitted answer files. The contract is not entirely secure, and I was unable to get a few "requires" working in time, most notably that the finally submitted IPFS file is a match to the promised one.

To be economically sensible, this project would require deployment on a scaling solution such as Polygon or Polkadot.

Challenges we ran into

Just about everything! This was my first foray into blockchain and smart contract development so it was a massive learning curve.

Smart contracts do not make good databases. An earlier version of this project hoped to store the IPFS hashes of tasks within the contract itself.
The web3 front-end ecosystem, and Solidity as a whole, is rapidly changing and information becomes outdated quickly.
Chainlink can be quite challenging to test, requiring deployment on a Testnet.
Solidity is very challenging to debug.
I was not able to find a node operator who could set up the External Adapter for my on-contract IPFS integrations in time for the end of this hackathon.

Accomplishments that we're proud of

I'm just happy to be here. This was a very ambitious project for one guy who had no idea what he was doing. I did not even discover "merkle proofs" until it was nearly April! I only knew which problem I wanted to solve. Having this contract functionally deployed on Kovan, and operable with my front-end (however cobbled together it is) is a big win for me.

What we learned

Everything in this project that is not React. Smart contracts are powerful, but come with their own limitations. However, limitations breed creative solutions!

IPFS is super cool, but currently slow and unwieldly as a "plug and play" solution for a live app.

What's next for Sozu

I believe there is a tremendous amount of potential to be explored in using smart contracts in the delegation and management of crowdsourced labour. I would like to create a user-friendlier UI, secure the contract properly, finish integration with IPFS and begin exploring other types of work that the system can be used for.