nex

A fully distributed package manager for the Web 3.0 era — - Trust only the package author, not any central authority. - Immutable, verifiable copies from peers that builds network resilience.

Comment 1

Inspiration

Do you think twice about security before installing an apt package? What about downloading an OS ISO image from an official site? The robust and trustworthy community of modern package managers, fueled by active open-source contributions, seems unshakably strong.

Except it isn't. "I suggest you find a different community to do experiments on," responded Linux Foundation Fellow Greg Kroah-Hartman. In 15 days, white-hat hackers from the University of Minnesota have caused the institution's permanent ban from the Linux kernel, despite purportedly well-intended experiments to identify current system weaknesses.

Central authorities, benign or malignant, are susceptible to too many external influences that weaken, if not undermine, the public trust. This is the central question that Web3 addresses: how do we interact and live in an increasingly centralized digital world when trust is repeatedly eroded?

What it does

Introducing nex, a unique, *nex*t-generational decentralized package manager that reduces unconditional trust to only the package author. Built with Iroh, a Rust-powered, performant platform that iterates from IPFS, nex removes any centrality in the process, making it ever more resilient to both censorship, hijackings, and network partitions.

How we built it

At its core, Iroh provides primitives — practically immutable ones in particular — for "[r]ealtime, multiwriter key-value sync" in the form of documents. A read ticket uniquely identifies and provides read access to all key-value pairs. A package author, the sole user with write access, adds metadata, versions, and hash-addressed file blobs that anyone anywhere can see and download. The design of Iroh inherently makes nex:

  • A minimal-trust framework: Only the package author controls the document/package. If not necessary, why delegate trust to anyone else?
  • A robust network: Hash-addressed, immutable blobs can be stored and served by anyone. Hijackings, censorships, and other attacks looming over centralized package managers are easily identifiable and circumventable.

image Illustration from Iroh documentation.

On top of the core CLI tool, we built a lightweight UI wrapper with Tauri (Next.js + Rust) that provides an intuitive and beginner-friendly means of getting into nex.

Challenges we ran into

Deliberating the Protocol. For Nex to be useful, it must prove resilience beyond current solutions. The active, heated debate about potential vulnerabilities, along with trade-offs between speed and reliability, was challenging to navigate in the time-sensitive setting of a hackathon. Sleep deprivation didn't help; at 3 a.m., we seriously considered reinventing TCP/IP before realizing coffee isn't a substitute for good judgment.

Unavoidable (De)serialization. Building the Tauri frontend, which hinges on the communication between the Next.js frontend and the Rust backend, mundane, trivial problems like (de)serialization became huge time sinks that we had not expected. It's amazing how, after 24 hours without sleep, even parsing JSON starts to feel like deciphering ancient hieroglyphs.

Trading Perfection for Completeness. Within the narrow timeframe of PennApps, it was difficult but necessary to find a delicate balance between attention to detail and complete functionalities. We focused on the latter, delivering the three core functionalities of package publication, package search by name, and installation. After all, perfection is hard to achieve when you're powered by energy drinks and three hours of sleep!

Accomplishments that we're proud of

  • Elegant protocol designs leveraging mature open-source frameworks.
  • Logarithmic time complexity in package distribution.
  • Intuitive, beautiful frontend with native Rust code.

We had quite some fun thinking ideally and theoretically, revisiting high school math. Suppose a user publishes a package that $n$ people simultaneously want to download. While Iroh limits each client to at most $p=5$ concurrent peer connections, it turns out the time complexity of distributing the package to all $n$ peers is logarithmic.

Initially, 1 person distributes the package to $k$ peer. After one such turn, $1+p$ clients can simultaneously serve the package, each to $p$ new peers. For a general $i\in\mathbb N_+$, let $a_i\in\mathbb N_+$ denote the number of clients who receives the package at turn $i$, with $a_1=1$ initially. Denote $S_k=a_1+\cdots+a_k$, so $a_k=S_{k-1}\cdot p$. Then, $S_k=S_{k-1}\cdot p+S_{k-1}$, and we see that $a_1=S_1=1$ implies $S_k=(1+p)^{k-1}$. Then, time complexity is evidently $O(\log_{1+p}n)=O(\log n)$.

What we learned

Software development does not exist in a socially void milieu; its meaning and value is precisely reflected in the ways people interact with such software. We set out with a lofty goal, to revolutionize how we leverage everyday programming and computer tools. Purely technical discussions regarding bandwidth, availability, and efficiency are far from the whole picture, and we're grateful for this valuable lesson learned from this intense yet meaningful hacking experience.

What's next for nex

The frontend admittedly lags behind the CLI in terms of functionality, but it offers drastically more user-facing advantages in and potential for visualization and interactivity. Our primary goal will be to catch up to the completeness of features offered by the CLI, while maintaining its unique benefits.

Meanwhile, we want to facilitate community engagement and contribution, a hallmark of open-source software that made nex possible and that nex proudly stands for. We hope to talk with various stakeholders, work with industry experts and grassroots enthusiasts, and explore the future of nex in greater depth to benefit the landscape of software distribution tomorrow.

Built With

Share this project:

Updates