Quine

Inspiration

I want to commoditise "developers" by allowing them to sell their time to the highest bidder whenever they want. People with programming skills should be able to work whenever they want at the real market rate.

A key issue to achieve this is to be able to "price" dev skills. Pricing of dev skills can be done algorithmically via data-driven approaches.

What it does

Estimates market demand for each Python library. It also estimates how strong is the supply for each of these libraries.

At this point, I implemented a very simple model that assumes that dev's value is solely determined by the number of libraries s/he uses.

It continuously clones open-source repos and retrieves contributor's number of adds/deletes to each

How I built it

Model assumptions:

• There's only one programming language in the economy: Python.
• Developer skills are solely determined by the libraries they know how to use.
• A developer knows how to use a library if he/she has edited a file that imports it.
• Market demand for libraries is perfectly correlated with the frequency library appears in the dataset I scraped.

1. Looked for a dataset of open source repository links.
2. Explored, the dataset and selected repos that had at least ten lines of Python.
3. For each GIT link: 3.1 I cloned the repo. 3.2. For each repo: + I filtered out Python files. + Computed how many additions/deletions each of the repository contributors had in each Python file. + Extracted all "import" statements to quantify which libraries are present in each file + Inserted data into a database.
4. Built dashboard to visualise results

Challenges I ran into

Dealing with very large datasets. ~15TB in total. I had to write code to do all the parsing.

Accomplishments that I'm proud of

Way to model the problem. Getting to build dataset, clean it, design database, and deploy a dashboard without other teammates :):

What I learned

There is a LOT of data hidden in GIT repositories.

What's next for Quine

Defining an MVP and build it. Put it up on Reddit.

Built With

• dash
• python
• react
• sql