Inspiration

Web3 infrastructure is powerful, but operating it reliably is still a difficult distributed-systems problem. A bad configuration can turn a healthy validator network into a latency, throughput, or consensus incident—and simply letting an AI "optimize" production infrastructure is not a safe answer.

We wanted to build something different: an AI-powered SRE control loop that can reason about blockchain infrastructure while being capable of saying "no."

That led to ChainPilot: a system designed to detect incidents, investigate possible causes, propose optimizations, challenge risky changes, verify candidates against SLAs, and recover when a change fails.

The core idea was simple:

Don't give AI unrestricted control of the chain. Give it the ability to reason, test, challenge, verify, and recover.


What it does

ChainPilot acts as an intelligent operations layer for Web3 infrastructure.

It continuously connects the operational loop:

Detect → Diagnose → Optimize → Challenge → Verify → Apply/Rollback

Its workflow includes:

  • Monitoring blockchain/network health and detecting abnormal behavior.
  • Optimization of configuration candidates against performance objectives.
  • Risk auditing that challenges proposals that could threaten consensus or violate safety constraints.
  • Shadow evaluation so promising candidates can be tested without blindly deploying them.
  • SLA verification against measurable targets for throughput, latency, and success rate.
  • Rollback/recovery when an applied change fails verification.
  • Explainability so operators can understand what happened, why a decision was made, and what the system recommends next.

One of our most important demo scenarios intentionally ends in a veto: an attractive optimization improves the candidate metrics but still fails the required SLA and creates an unsafe consensus margin. ChainPilot blocks it rather than pretending the optimization worked.


How we built it

We built ChainPilot as a combination of Web3 infrastructure, observability, optimization, agentic reasoning, and safety controls.

The blockchain layer uses a Hyperledger Besu / IBFT2 consortium topology, while the control layer evaluates network telemetry and configuration candidates.

The system is organized around specialized responsibilities:

Blockchain telemetry
        ↓
     Monitor
        ↓
    Optimizer
        ↓
  Risk Auditor
        ↓
 Shadow / Candidate
    Evaluation
        ↓
   SLA Verification
      ↙       ↘
   PASS       FAIL
    ↓           ↓
  Apply      Reject /
             Rollback
        ↓
    Explanation

Rather than allowing an optimization model to directly change everything, we introduced policy and verification gates between proposal and execution.

We also built a visual SRE war room to make the entire decision process observable—from incident detection to agent reasoning, candidate evaluation, vetoes, verification, and recovery.


Challenges we ran into

The hardest part wasn't making an optimizer produce a better number.

It was making the system safe enough to trust.

1. Optimizing blockchain infrastructure is a constrained problem

Increasing throughput isn't useful if it causes consensus instability or destroys latency.

A configuration can look better in one metric while making the overall system worse.

We therefore had to reason about multiple constraints simultaneously rather than optimizing a single number.

2. AI should not automatically get the final say

One of our biggest design challenges was avoiding the classic:

AI proposes → AI deploys → hope it works.

We introduced a separate risk/challenge stage so a proposed optimization can be rejected before it reaches the network.

3. Demonstrating failure is harder than demonstrating success

A polished demo naturally wants every experiment to succeed.

But infrastructure engineering doesn't work that way.

We deliberately built the demo around a candidate that should be rejected. That forced us to clearly distinguish baseline metrics, candidate metrics, SLA requirements, veto decisions, and production state.

4. Keeping the evidence honest

We had to be extremely careful about distinguishing:

  • measured results
  • shadow/candidate results
  • production results
  • rollback results
  • mathematical projections

That discipline became part of the product itself.


Accomplishments that we're proud of

We're especially proud that ChainPilot doesn't treat autonomy as "AI gets unrestricted access."

Instead, autonomy is structured around guardrails and evidence.

A few things we're particularly proud of:

  • Built an end-to-end AI-powered Web3 SRE control loop.
  • Integrated monitoring, optimization, risk analysis, verification, and recovery into one workflow.
  • Demonstrated a Risk Auditor vetoing a risky candidate before deployment.
  • Added explicit SLA verification rather than judging an optimization by a single metric.
  • Demonstrated candidate optimization where throughput improved from 11.9 TPS to 17.7 TPS, while correctly recognizing that the candidate still failed the stated SLA.
  • Made the system explain why a candidate was accepted, rejected, or rolled back.
  • Built a judge-friendly visual war room that makes the normally invisible infrastructure decision process understandable.

Most importantly, we made failure a first-class outcome instead of hiding it.


What we learned

We learned that autonomous infrastructure isn't primarily an AI problem.

It's a systems engineering + safety + observability problem.

A sophisticated model can propose a configuration, but that doesn't mean the configuration is safe. The surrounding system needs measurable constraints, independent checks, controlled experimentation, verification, and recovery.

We also learned that a good autonomous system needs the ability to refuse an action.

In our case, the Risk Auditor is not there to make the demo look intelligent. It's there because an optimizer can find a configuration that looks attractive numerically while still being unacceptable operationally.

Finally, we learned that transparent failure is powerful. Showing a candidate fail—and showing the system correctly prevent that failure from reaching production—is arguably more convincing than showing an unrealistic "AI optimized everything perfectly" demo.


What's next for ChainPilot

The next step is moving from a strong prototype toward a production-ready Web3 operations platform.

We're focusing on:

  • More real-world Besu validator telemetry and fault injection.
  • Stronger persistent incident and decision history.
  • More rigorous policy enforcement and configuration bounds.
  • Fully reproducible benchmark and evaluation pipelines.
  • More sophisticated rollback and recovery strategies.
  • Human approval workflows for high-risk changes.
  • Multi-network and multi-consortium support.
  • Long-running autonomous operations with audit trails.
  • Learning from historical incidents to improve future recommendations.

Our long-term vision is an intelligent reliability layer for decentralized infrastructure—one that doesn't just tell operators that something is wrong, but can investigate, challenge its own decisions, safely test alternatives, and recover when things go wrong.

The goal isn't to make AI control the blockchain. The goal is to make blockchain infrastructure safer to operate with AI.

Built With

  • agents
  • ai
  • autonomous
  • besu
  • blockchain
  • devops
  • distributed
  • hyperledger
  • ibft2
  • llm
  • multi-agent
  • observability
  • optimisation
  • python
  • react
  • web3
Share this project:

Updates