Rapid Relay

Rapid Relay ensures fast, secure file transfer over unstable networks using chunking, encryption, multi-path routing, and erasure coding—ideal for critical data like medical scans or disaster plans.

Comment

Inspiration

We are constantly crippled by the last mile problem: sending massive, mission-critical files over unstable networks (rural clinics, remote engineering sites, disaster zones). Existing solutions fail under pressure, forcing complete restarts and delaying vital decisions. We were inspired to build a guaranteed delivery system—a truly smart logistics protocol that ensures data integrity and arrival speed, no matter the link quality.

What it does

Rapid Relay is the Top Smart File Transfer System. It guarantees fast, resilient, and secure file delivery by:

  • Priority Channels: Urgent files preempt low-priority traffic instantly.
  • Adaptive Speed: Dynamically adjusts chunking and parallelism to optimize throughput on unstable links.
  • Multi-Path Routing: Splits data across secure, reliable relay nodes to bypass the most fragile network segments.
  • Local Reconstruction: Uses Erasure Coding to mathematically rebuild lost or corrupted data locally, eliminating the need for costly network retransmissions.
  • End-to-End Integrity: Confirms $100\%$ data fidelity via per-chunk and master file cryptographic hashing.

How we built it

We implemented a robust architecture using multi-threaded pools to separate I/O and CPU-intensive tasks.

  1. Preparation: Files are chunked, encoded with Reed-Solomon Parity, and AES-256 encrypted.
  2. Transmission: Dedicated Network Sender Threads saturate the available bandwidth while the Control Thread monitors link quality using real-time RTT measurements.
  3. Resilience: Corrupt chunks are detected instantly via hash checks. The system prioritizes local repair using parity chunks before resorting to retransmission.
  4. Security: All data remains encrypted on the temporary relay nodes, with decryption possible only at the final destination.

Challenges we ran into

Our biggest hurdle was balancing complexity with speed. Initially, we faced two critical failures:

  1. Ineffective Prioritization: Our first priority model was flawed, attempting to prioritize based on file size (smaller files win). This failed to meet the mission-critical requirement, as a large, urgent file was constantly blocked by many small, low-value files. We successfully pivoted to a Weighted Priority Queue system that honors assigned criticality, ensuring the $10\text{ GB}$ trauma scan jumps the line every time.

  2. Unworkable Backup Idea: We explored a "pointer-based" approach for data repair, which would have required retransmitting broken chunks across the unstable link. This was a latency killer. We replaced this with mathematical Erasure Coding, which allows for local, near-instantaneous reconstruction of lost data, making the system truly resilient and fast.

Accomplishments that we're proud of

We built a system that maintained an average transfer success rate above $99.99\%$ even when simulating network dropouts and $30\%$ packet loss—a remarkable feat for a software protocol. We are especially proud of the seamless integration of Erasure Coding, which successfully reconstructed large data volumes locally, demonstrating superior resilience compared to traditional retry mechanisms. Finally, achieving true low-latency priority channels means our solution is ready for real-world, life-or-death scenarios.

What we learned

We gained a deep appreciation for the trade-offs between resilience overhead and raw speed. We learned that for "unstable links," adding initial overhead (like parity generation and sophisticated scheduling) is a worthwhile investment, as it saves exponentially more time by avoiding transfer failures and lengthy retransmissions. Intelligence in scheduling is more valuable than brute force parallelization.

What's next for Rapid Relay

We plan to implement a dynamic cost-analysis model to automatically choose between direct transfer and multi-node relay routing based on a calculated score of link stability and cost. We also intend to develop a lightweight client SDK for easy integration into specialized hardware, such as mobile satellite terminals and remote server clusters, solidifying Rapid Relay as the industry standard for guaranteed data delivery.

Built With

Share this project:

Updates