IPFS / webnn Huggingface Bridge

I am porting huggingface transformers libraries to webNN, creating a virtual fileystem, and creating a decentralized multi-agent architecture (aka swissknife which is under DMCA strike by Anthropic)

Comment

Inspiration

I started building this infrastructure after a meeting at the Linux Foundation Open Platform for Enterprise AI in May 2024, despite the fact the linux foundation didn't decide to implement my plan, I was encouraged by Brian Behelendorf to continue doing working on it.

What it does

I am providing access three MCP servers:

1) a virtual filesystem based on the IPFS standard, with graphrag, adaptive replacement cache, and 10 filesystem backends 2) a datasets manipulation MCP server, that provides tools for archiving websites, analyzing financial data, formal logic proofs, etc. 3) a Huggingface model server, that runs in python on 6 hardware backends (cpu, cuda, openvino, mps, rocm, qnn), and also supports webnn / webgpu

Not Provided: Swissknife multi agent system (DMCA'd by Anthropic because i forked the "Anon-Kode" 4chan reverse engineer of claude code, upon which I added half a million lines of code on top of, despite reverse engineering not being copyright infringement, and what they need is a patent rather than a copyright)

SwissKnife is a powerful, terminal-based AI toolkit built entirely in TypeScript for the Node.js environment. It provides a unified interface to interact with various AI models, manage complex tasks, interact with decentralized storage (IPFS), and extend capabilities via the Model Context Protocol (MCP).

Key Features

  • Unified TypeScript Architecture: A cohesive system built entirely in TypeScript, integrating AI, task management, storage, and CLI components. Follows clean room principles based on original Goose concepts. See docs/UNIFIED_ARCHITECTURE.md.
  • Advanced AI Agent: Features sophisticated reasoning, tool usage, and memory management.
  • Graph-of-Thought (GoT) Engine: Enables complex problem decomposition and non-linear reasoning paths for advanced tasks.
  • Enhanced TaskNet System: Includes a high-performance Fibonacci Heap scheduler for dynamic task prioritization and Merkle Clock coordination for potential distributed task execution.
  • ML Engine Integration: Supports local ML model execution with hardware acceleration detection (via Node.js bindings like ONNX Runtime).
  • Virtual Filesystem (VFS): Provides a unified interface over multiple storage backends, including local filesystem and IPFS.
  • IPFS Integration: Leverages content-addressable storage via an IPFS client (connecting to IPFS Kit MCP Server or other IPFS nodes) integrated into the VFS.
  • Rich Terminal UI: Consistent command structure, interactive prompts, progress indicators, and formatted output (including tables, JSON, YAML). Uses Ink/React for some components.
  • Model Context Protocol (MCP): Can act as an MCP server and includes services for managing MCP server connections and tools.

Built With

Share this project:

Updates