sky

Inspiration

The explosion of Large Multimodal Models (LMMs) and complex distributed workflows has pushed modern infrastructure to its limits. Traditional schedulers treat compute nodes as static, homogeneous pools. However, in heterogeneous hybrid-cloud and edge environments, resources are fragmented, networks are dynamic, and workloads exhibit complex topological dependencies.

We asked ourselves: Why should intelligent applications run on dumb, static infrastructure? This inspired us to build sky—a next-generation intelligent resource scheduling framework designed to bridge the gap between high-level application intents and low-level physical topologies.

What it does

sky is a high-performance, topology-aware resource scheduling engine designed for large-scale training and inference across hybrid-cloud and edge networks.By utilizing a customized Graph Neural Network (GNN) encoder, sky transforms heterogeneous cluster topologies and workload dependency graphs into a unified vector space. It dynamically solves a multi-objective optimization problem to minimize both scheduling latency and cross-cloud bandwidth costs.Formally, given a workload deployment request, sky optimizes the policy $\pi$ under strict resource constraints:$$\min_{\pi} \sum_{i=1}^{N} \left( \alpha \cdot T_{\text{delay}}(i) + \beta \cdot C_{\text{network}}(i) \right)$$Where $T_{\text{delay}}$ represents the end-to-end execution delay, and $C_{\text{network}}$ represents the inter-node communication cost under network jitter.

How we built it

We built sky using a highly decoupled microservice architecture:

The Core Scheduler (Python/Go): Implements the scheduling algorithms and interfaces with container orchestrators (like Kubernetes).

Topological GNN Encoder: Uses a custom GNN-based state representation model to capture complex dependency graphs of distributed tasks.

High-Fidelity Simulator: Developed an event-driven simulator capable of mocking millions of scheduling requests in a cloud-edge topology within seconds, allowing us to rapidly iterate policies.

Challenges we ran into

The Curse of Dimensionality: Resource scheduling in heterogeneous networks is a classic NP-hard problem. Standard mathematical solvers fail to output decisions within sub-millisecond limits when nodes scale to $10^4$ or higher.Network Jitter & State Inconsistency: In a hybrid-cloud environment, network latency changes dynamically. Maintaining a consistent global state of resources without introducing massive synchronization overhead was a major algorithmic hurdle.

Accomplishments that we're proud of

We successfully reduced the scheduling decision overhead to under 10 ms for complex topologies containing thousands of tasks.SchedulerDecision Overhead (ms)Network Cost ReductionGPU UtilizationK8s Default~15 ms0% (Baseline)68%DRF (Classical)~35 ms12%74%sky (Ours)< 10 ms42%89%

What we learned

We realized that in modern distributed systems, perfect planning on paper is useless if the scheduling overhead itself exceeds the execution savings.

Designing an elite scheduler is not just about writing complex heuristic algorithms; it is a delicate balancing act between prediction accuracy, decision latency, and system robustness under volatile real-world conditions.

What's next for sky

Production-ready Kubernetes Plugin: We are working on wrapping sky into a native scheduling framework plugin for K8s, enabling direct deployment in production clusters.Online Reinforcement Learning: Integrating a continuous online learning loop to adaptively adjust the weighting coefficients ($\alpha, \beta$) based on historical scheduling drift.Open Source & Academic Paper: Preparing to open-source the core simulator engine and draft a technical paper targetting top-tier systems conferences.

Built With

• platform