We were looking for what challenge to build for and decided to stay with the most technical one, offered by GenX (https://genx.global/). The company is providing a distributed system for companies to share their fragmented data on it. For the challenge, they asked teams to build a scalable p2p network, which would allow different nodes (companies) to communicate with each other.
What it does
The system, consisting of worker nodes and controller node, allows worker nodes to communicate with each other by sharing data in a peer-to-peer manner (similarly as in torrent network). Thus, companies can keep their data secure but be able to share and receive a fraction of it.
How we built it
Implemented Azure Architecture
- Every node (VM) in the network has its own Virtual Network (VNet) which is peered to each other
- All nodes communicate through gateways and VPN
- A Load Balancer for the main controller Node
- Auto Scaled brought up VM Machine Sets in unique Nodes for intensive CPU computation tasks
- Azure Dashboards with CPU / Bandwidth / IO Ops
- An endpoint to fetch all the VM IP's from a specific Azure Resource
Challenges we ran into
- Figuring out on how to make proper and cheaper p2p network on Azure (alternatives were to use Kubernetes, or Blockchain)
- Learning Azure
Accomplishments that we're proud of
- Learning Azure from scratch and actually building something cool on it!
- Being a team of two, having built fully working MVP with no hack (that's rare, haha!)
What we learned
- Virtual Networks, Virtual Network Peering, P2P communication
- Protocols (gRPC, etc.) and additional security layers (Intel SGX)
What's next for Cloud Madness (HackMoscow 2019)
Probably working with GenX on developing the solution further. Will discuss that over the beer ;)