GraphFlow-RL

Project Proposal: GraphFlow-RL

Team: Quepaxa

Track 2: Supply Chain

Concept Focus: Sim2Real Adaptive Assembly Sequence Planning & Stochastic Disruption Management

Motivation

Modern infrastructure projects rely on off-site prefabricated and precast concrete components to compress construction timelines. However, these systems depend on strict, just-in-time logistics. Because construction sites operate under constant, random real-world disruptions, such as traffic congestion, weather anomalies, and handling delays, a single late delivery triggers a cascading freeze across the entire assembly floor, idling multi-million-dollar assets like heavy cranes and specialised crews.

GraphFlow-RL resolves this bottleneck through a Sim2Real (Simulation-to-Real) pipeline. By pairing a programmatic discrete-event simulator with an advanced Graph Attention Network-driven Reinforcement Learning (GAT-RL) policy engine, our solution moves the industry away from slow, reactive scheduling guesses toward proactive, millisecond-level optimal adjustments. Trained via aggressive domain randomisation, GraphFlow-RL transforms data-poor job sites into highly resilient, automated logistical systems.

Problem Definition

The infrastructure sector faces two structural bottlenecks that prevent traditional AI models from successfully optimising job site logistics:

Lack of Refined Data: Machine learning typically relies on pristine, historical logs. The construction industry has the exact opposite: micro-operational logs are fragmented, unstandardized, or locked away by private contractors. No large-scale, real-world dataset is available to train a standard predictive model.

The Failure of Static Mathematics: Real-world sites operate under volatile uncertainty. When a delivery truck carrying a critical structural column is delayed by 45 minutes, traditional project management frameworks such as the Critical Path Method (CPM) break down. Alternative search models, such as Genetic Algorithms, take minutes or hours to recompute a fresh schedule, which is too slow to use dynamically on an active site.

As a result, managers are forced to make reactive guesses. This causes supply trucks to queue up outside gates (incurring expensive demurrage fees) while high-cost machinery sits completely idle waiting for out-of-order components. Proposed Solution

Instead of relying on historical field logs, GraphFlow-RL uses Domain Randomisation inside a highly volatile, physics-constrained digital twin. We build a programmatic simulator that intentionally exposes our AI agent to millions of simulated worst-case site variations.

The Technical Workflow

Stochastic Simulation (SimPy): The project site is built as a discrete-event simulator using SimPy. Real-world uncertainty is modelled directly within the system: logistics transit times follow Log-Normal distributions, crane availability transitions follow Markov Chains, and human assembly speed follows Beta distributions.

Topological Inductive Graph Encoding (GAT): A project schedule is a network of dependencies, not a flat table. We represent the site state as a Directed Acyclic Graph (DAG), where nodes represent components or tasks and edges denote physical construction dependencies. A Graph Attention Network (GAT) compresses this network topology into dense latent vectors.

Action Masking & Valid Action Sampling: To prevent the AI from attempting physically impossible actions (e.g., placing a roof deck before the structural walls are anchored), an integrated action-masking layer filters out invalid choices. The agent only evaluates viable operations at that exact millisecond.

Policy Execution: A Deep Reinforcement Learning agent utilising MaskablePPO ingests the topological graph embeddings and instantly selects the next optimal logistical choice, such as re-routing a transit vehicle or shifting crane priority.

The Mathematical Optimisation Engine

The reinforcement learning agent optimises its sequencing decisions by maximising a multi-objective reward function at every step t: Rt = - ( Tidle + M + Cholding ) Tidle: Idle time cost of critical, high-lease machinery on-site. M: The net change in the projected total project completion time (makespan). Cholding: Logistics penalties and demurrage costs from delivery vehicles queuing outside gates.

Originality & Technical Feasibility

Embracing Partial Observability

GraphFlow-RL solves missing real-world data by formulating the environment as a Partially Observable Markov Decision Process. During the simulation phase, we implement Stochastic Graph Dropouts (pdrop = 0.20), randomly deleting 20% of the nodes and edges. This forces the neural network to make resilient decisions based on local topological neighbourhoods, ensuring that if a field log is skipped or a telemetry update drops out, the system continues to generate valid operational decisions.

Market Differentiation & Scalability

Most construction supply chain projects rely either on outdated static charts or superficial chatbot wrappers around Large Language Models. LLMs frequently hallucinate structural order and fail at mathematical combinatorics. GraphFlow-RL shifts the entire computational burden from the job site to the cloud training environment:

Operational Metric Legacy CPM Software Metaheuristic Engines GraphFlow-RL Solution Reaction Delay Manual intervention (Hours) Iterative search lag (Minutes/Hours) Instantaneous (Milliseconds) Data Requirements Total manual baseline entry Rigid, pristine data models Synthesised via Domain Randomisation Scaling Complexity Exponentially slower as the project grows Suffers from a combinatorial explosion Scale-invariant via local sub-graph attention

Built for Universal Scale

Because we utilise inductive Graph Attention Networks rather than fixed matrix calculations, the model does not lock onto a single structural layout. It learns relational behaviours between adjacent structural elements.

This enables Universal Scalability: a policy trained on a simulated 5-story building can be deployed immediately onto a massive horizontal infrastructure project without requiring a full neural network retraining cycle. It simply analyses local task dependencies and optimises them seamlessly.

Built With

• ml