OneStorage

Create, buy or sell data.

Comment

Inspiration

Since our college days, I have participated in competitions and even worked on projects that would require datasets, but it was tough to find them. One of the main reasons was that there was no incentive for people to collect them or make them public. On top of it, the sources were fragmented. There is no ordinary place where you can search or download data. Secondly, there is an inherent problem with the data sold; there is no accountability or way to track data, who got it from whom or who sold it to whom. Lastly, there is no way to collaborate where one creates the data, and anyone who has to write permission can update the dataset.

So, I made it my mission to solve these problems. As I was looking for the solution, I came across NFT. It partly solved the problem, but it was enough to inspire me to create a Shareable Token that could be shared among people. Furthermore, the EdgeStore allowed us to store a large amount of data on CDN that could do wonders when combined with Shareable Token.

What it does

OneStorage is a platform where anyone can create a dataset using GraphQL for writing schema. The schema is then parsed to get data types for checking constraints when a user pushes a row or rows, column names and data skeleton. Once the dataset is created, we use GraphQL language to mutate or query the dataset. After the dataset is populated with data, we can save it on the EdgeStore network. Afterwards, we can also share the dataset on the marketplace, where anyone can buy different permissions, such as read permission and read-write permission. If the user buys read authorisation, he cannot update the contract, and for read-write authorisation, they can also edit the contract. To make the data less heavy on the bandwidth, I have created a mini blockchain with a chunk of 1024 rows. Moreover, my app allows having a filesystem that closely emulates UNIX.

GraphQL

How to write schema?

There is no need to learn a new language to write schema. We use the GraphQL schema language. Moreover, we use this schema to parse types and use it to check constraints or types.

Mutations

There are three mutations available:

1. [addRow]

The function adds a row to the dataset.

Example:

mutation {
    addRow(input: { /* ColumnName : Value, ... */})
}

Returns : true if row is added successfully.

2. [addRows]

The function adds multiple rows to the dataset.

Example:

mutation {
    addRows(input: [
        { /* ColumnName : Value, ... */},
        {...},
        ...
    ])
}

Returns : true if rows are added successfully.

3. [commit]

The function commits the uncommitted chunk to the EdgeStore.

Example:

mutation {
    commit
}

Returns : true if commit was successful.

Query

There are three mutations available:

1. [show]

The function fetches the already in cached chunks and displays them as specified by the users.

Example:

{
    show{ 
        /* ColumnName, ... */
    }
}

Returns: Array of Objects containing columns that the user has asked for.

2. [info]

The function gives the dataset info.

Info Schema


type Fields {
    name: String!
    type: String!
    description: String!
}

type Info {
    description: String!
    path: String!
    name: String!
    size: Int!
    source: String!
    height: Int!
    fields: [Fields]
}

Example:

{
    info{ 
        name,
        fields: {
            name,
            type
        }
    }
}

Returns: Object containing data that the user has asked for.

3. [loadChunk]

The function loads chunks in the given range.

Prototype

type Query {
    loadChunk(start: Int, size: Int): /* Table */
}

Example:

{
    loadChunk(start: 0, size: 3) {
        /* ColumnName, ... */
    } // [0, 0 + 3)
}

{
    loadChunk(start: 3) {
        /* ColumnName, ... */
    } // [3, 3 + inf)
}

How we built it

The backbone of our project is Theta EdgeStore, Thetanet with Ethereum RPC adapter, Custom Token--Shareable Token-- and Firebase. Other than that, for:

Front-end: HTML, Javascript, Vuejs, Vite, Typescript, Web3, Quasar, GraphQL, FontAwesome, Material Icon and SASS Back-end: Nodejs, Typescript, Solidity Compiler (Solcjs), and GraphQL

Shareable Token

The token is similar to NFT but not a replacement. The token has a maximum of 256 levels of access, with 0 being no access. When you buy access, you do not have to pay extra for the next level of access because it will be the difference between the prices of the next level and the current level. Once a person creates a contract with set prices for the levels that he/she chooses to be, sharing can take place by buying the access.

Using the Shareable Token, I have created a Shareable Storage with two access levels: read access and read-write access. Once a person buys read level, he/she can only read the dataset but never can they update the contract. On the other hand, read-write access allows updating the contract where the Edgestore's content address is stored. To optimise the gas for storing the data on the contract, I have been only storing the CDN address on the contract rather than the dataset's schema and address.

Filesystem

I have created a filesystem heavily inspired by UNIX to store data online in a directory form. You can navigate and easily create folders to separate workspaces that may contain different data based on your use.

Firebase

Firebase allows us to have users and store their information in persisting storage. It also keeps the contract's address with additional information for the marketplace. It also contains the user information, such as name, avatar, email address, and serialised filesystem that has information about the structure of datasets and folders.

Challenges we ran into

The main problem that persists is not being able to deploy the app online because of the EdgeStore. And also not being able to test the app in the wild.

Accomplishments that we're proud of

I have many things to be proud of, but a few jump out the most.

  • Able to develop a new token and implement it in an app.
  • I created a UNIX a filesystem.
  • I learned about Contract within one day from a guy who did not know anything about it before the hackathon and implemented a whole new token without any help from outside.
  • I created a blockchain to reduce data bandwidth for a dataset.
  • I learned how blockchain works and how to implement one.

What we learned

  • I have developed a deeper understanding of Theta's RPC interface.
  • I also learnt about the contract, which I previously was oblivious to.
  • I developed a more profound knowledge of typescript and GraphQL, from parsing to using in the front-end.

What's next for OneStorage

There are a few features that I would like to have:

  • Add support for images and videos
  • Ability to merge data
  • Notifications if the contract is updated.
  • Ability to track the data with history.

More Information

Name: Amit Singh

Email: amitsingh19975@gmail.com

Role: Front-end and Back-end developer

Track: DeFi

Built With

Share this project:

Updates