About Ghost Order BookWhat Inspired UsLet’s be honest: if you’ve ever placed a sizeable order on an open-book DEX, you know that subtle sinking feeling when the price slips against you before your transaction even gets mined.In traditional finance, institutional traders use dark pools for a reason—not to be shady, but to avoid broadcasting their hand to predatory arbitrageurs. On public blockchains, transparency is usually praised as a feature, but for active liquidity providers, it often acts like a tax. Every open limit order sits in plain sight, inviting MEV sandwich bots, front-runners, and toxic order flow that systematically drains alpha.The realization was simple: Transparency without privacy isn't fair execution—it's exposure.We set out to fix this friction. We wanted to build an execution layer that delivers the sub-millisecond calculation speeds of traditional quantitative trading desks while guaranteeing that private order logic stays strictly off-ledger.How We Built ItWe built Ghost Order Book by combining hardware-level vector processing with zero-knowledge cryptographic state updates.┌─────────────────────────┐ 100Hz WebSockets ┌──────────────────────────┐ │ Rust Axum Engine │ ────────────────────────> │ Next.js HFT Dashboard │ │ (C++ AVX-512 Kernel) │ │ (Real-time Telemetry) │ └───────────┬─────────────┘ └──────────────────────────┘ │ │ ZK Commitments & Nullifiers v ┌─────────────────────────┐ │ Midnight Compact Engine │ │ (Off-Ledger Proofs) │ └─────────────────────────┘
- Hardware-Accelerated Vector Engine (C++ & Rust)At the core of our backend is a native C++ kernel called from Rust via C-FFI. We leveraged 512-bit AVX-512 vector registers (Fused Multiply-Add) operating on 64-byte aligned Struct-of-Arrays (SoA) memory blocks to process order book depth at 100 Hz.The kernel calculates Volume-Weighted Average Price (VWAP) continuously:$$VWAP = \frac{\sum_{i=1}^{N} P_i \cdot Q_i}{\sum_{i=1}^{N} Q_i}$$It simultaneously derives the real-time Order Book Imbalance ratio ($I$):$$I = \frac{\sum Q_{\text{bid}} - \sum Q_{\text{ask}}}{\sum Q_{\text{bid}} + \sum Q_{\text{ask}}} \quad \text{where } I \in [-1.0, 1.0]$$2. Zero-Knowledge Circuit Layer (Midnight Compact)While the SIMD kernel tracks telemetry, private order execution runs through Compact smart contracts on Midnight. The circuit proves order validity—confirming that an secret order price $P_{\text{order}}$ satisfies $P_{\text{order}} \ge P_{\text{floor}}$ and quantity $Q > 0$—without revealing the price or volume to the public state.The proof generates a persistent commitment hash:$$\text{Commitment} = H(P_{\text{order}} \parallel Q \parallel \text{Nonce})$$Settlement consumes a unique nullifier to enforce atomic execution and prevent double-fills off-ledger.3. Institutional Telemetry Dashboard (Next.js)The frontend connects via WebSockets directly to the Axum broadcast channel, rendering live VWAP calculations, order imbalance percentages, and dynamic anti-frontrunning protection bounds.The Challenges We FacedBuilding at the intersection of raw low-level C++ memory management and cutting-edge ZK cryptography meant hitting some serious walls along the way:Memory Boundary Realities: Passing dynamic array data between Rust and C++ using AVX-512 instructions requires strict 64-byte alignment (alignas(64)). A single unaligned offset caused instant hardware segmentation faults (SIGSEGV) during high-frequency simulation runs. Fixing this required careful memory padding on Struct-of-Arrays buffers.Balancing Speed with Proof Generation: High-frequency trading operates in microsecond bursts, whereas ZK proof generation takes time. Harmonizing the 100 Hz telemetry stream with asynchronous off-ledger ZK state verification forced us to rethink state updates—streaming telemetry continuously while validating execution commitments asynchronously.State Management at 100 Hz: Streaming JSON payloads over WebSockets 100 times per second quickly choked React’s re-rendering pipeline. We had to carefully isolate telemetry state updates to prevent UI stutter during market shifts.What We LearnedPrivacy and performance aren't opposites. You don't have to sacrifice high-frequency responsiveness to achieve zero-knowledge privacy; you just need to place the computational boundaries in the right layer.Hardware-aware code still reigns supreme. Offloading repetitive mathematical reductions to 512-bit vector registers dropped kernel execution times significantly, freeing up CPU overhead for ZK serialization.Designing for institutional trust requires empathy. Traders don't just want privacy—they want proof that their trade was executed fairly without handing their strategy over to front-running bots.
Built With
- avx-512
- axum
- bash
- c++
- c-ffi
- cargo
- compact
- cryptography
- docker
- docker-compose
- lucide-react
- midnight
- next.js
- node.js
- quantitative-finance
- react
- rust
- simd
- tailwind-css
- tokio
- typescript
- websockets
- zero-knowledge-proofs
- zk-snarks
Log in or sign up for Devpost to join the conversation.