Ludds.io: Package Market. Encouraging Paid Software.

keybase chat with ludd

Inspiraton

I really tried to identify myself with the world of Open Source, however couldn't in the end and gave up. Having made over 200 npm packages, including a novel testing and documentation frameworks, I've discovered that people are simply not interested in my work. It seems that once cool, open-minded movement of techies has become a biased, entitled mass that thinks "software = free" and that's everything there is to it. After finally realising that nobody is going to support my work, I decided to do what's usually done to make a living: sell my software.

In Democracy, people are allowed to hold various informed opinions. I'd like to give people the choice of selling their packages. There are many, who wouldn't mind putting a price tag on a piece of code, which they thoroughly tested, documented and perfected during sleepless hours, but the industry doesn't give them a chance simply because there's no platform. When estimated, many would find themselves surprised that software that they upload online for free, costs at least a few thousand dollars, and can bring recurring revenue.

Open Source isn't really fair to everyone. Once a maintainer has captured the market with his free tool, he/she will dominate it simply because the tool becomes the standard, and nobody else receives a chance to showcase their great packages and build up the market share, too. The "free" model evolves into predatory pricing strategy, while it is the customer who's suffering when they cannot receive adequate customer service because the maintainers use excuses like "we're volunteers" even though they're not providing fair support plans either.

I don't want donations. I don't want community. I want to build, market and sell. I want to create a brand and earn reputation as a software vendor whom individual customers as well as companies love dealing with, because they know that I care. My view is completely antagonistic to the FOSS paradigm, and I believe that although the user might feel like she has the right to study and improve the program, the software publisher also has the right to keep the source closed as his intellectual property. That's why I'm passionate about building a package market.

What it does

The entry point to the app is two-fold.

First, package makers upload their work to the new registry (ludds.io). Using Keybase as the cryptographically sound client behind the package storage, the registry allows package makers to:

upload the indexes of packages (such as @scope/package-name) to their public folder at username.keybase.pub/packages so that other people are able to access those indexes and be sure that they were created by the publisher, and not tampered with. The download links to actual archives with distributed material are signed using npm's and yarn's shasum algorithm to ensure authenticity.
upload the actual packages to a private folder at keybase://private/username,ludd which is shared with the bot called ludd. Once uploaded, the bot moves the tar.gz archives onto its own private folder shared with kbpbot that will serve them from Keybase pages instance. The packages.ludds.io domain is configured to serve the files according to Keybase's documentation.

The security of packages is ensured by the fact that the index file is downloadable from the user's pub directory, visually identifiable as USERNAME.keybase.pub. As shasums are present, even though the domain that hosts the archives isn't transparent, the consumers are sure that the package was published by the real owner.

Secondly, those who wish to consume packages, need talk to the ludd chat bot, who'll process their requests. Users can search for packages using keywords and natural language (not implemented in hackathon) and once they found the suitable package, complete the purchase. Those who create an account for the first time, are sponsored with reserves enough to claim balances.

How I built it

By service package indices from Keybase pages.

Access Control

The access to the registry can be controlled with .kbp_config file where the password is hashed with bcrypt encoded string (username -> bcrypt-hashed password)

Docs on https://pkg.go.dev/github.com/keybase/client@v5.3.1+incompatible/go/kbfs/libpages/config

{
  "version": "v1",
  "users": {
    "user": "$2y$12$G9WEY55xbcxDp40xzTeQReT661MAI5qWUoKvsyQFkbEw5O2B5SEdW"
  },
  "per_path_configs": {
    "/": {
      "whitelist_additional_permissions": {
        "user": "read"
      },
      "anonymous_permissions": "list"
    }
  }
}

When the password is hello, it needs to be set as aGVsbG8= in .npmrc:

# .npmrc
https://packages.ludds.io/neddludd:_password=aGVsbG8=
https://packages.ludds.io/neddludd:username=user

CLI

There are 2 CLI programs that are used.

First, neoluddite is a piece of software that builds registry indexes from the local database that contains meta records about packages. This is outside the scope of this hackathon and was developed as part of the Luddites projects.

However, it was modified to adapt for the needs of the hackathon in the following ways:

The blank argument was added to enable to generate indexes without extension. GitLab, for example, could serve index.html at https://registry.gitlab.com/@scope/package/ appropriately, but Keybase will serve a redirect request to package/ which won't be handled by npm and yarn clients appropriately. This means that we can simply create @scope/package file with the JSON information about the versions that are then installed.
An additional tars option was added, to specify a distinct folder where the packed packages are placed. Previously, they would go in the public folder, but because the the indexes are now files and not index.html files inside directories, TARs need to go into a separate folder, as one cannot write to @scope/package.
Finally, in the previous version, the registry was maintained as a single repository where indexes would go into public/@scope/package/index.json and tars tars/@scope/package/version.tar.gz, which was then LINKED to public/@scope/package/~/version.tar.gz. As we're maintaining the tars separately, they need to be COPIED into keybase://private/{username},ludd/@scope/package/~/version.tar.gz which is then read by the Keybase bot and copied into keybase://private/ludd,kbpbot/@scope/package/~/version.tar.gz so that the package can be served from the global DNS-resolved address.

Challenges I ran into

Keybase doesn't provide the API for the FS methods. I have to run the bot in parallel to the main session where a real user is logged in. This was challenging as the KBFS service didn't start for the temp home dir, which wasn't documented.
When copying files using fs cp method, an unknown error would pop up, which was fixes by recursively making directories into which the file had to be copied.

Accomplishments that I'm proud of

Using Keybase client in an imaginative way as a registry for packages. Accepting payments for software.

What I learned

About Stellar's new features and how to work with the SDK. How to fund balances and why users can't use their new accounts straight away, as this requires reserves which can be sponsored by an anchor.