Kintsugi AI: Autonomous Self-Healing Cloud Infrastructure

Kintsugi AI is an autonomous platform using chaos engineering and LLMs to detect, diagnose, and automatically heal global cloud infrastructure failures in real time.

Comment

The Philosophy of Resilient Systems

Our project is inspired by the ancient Japanese art of Kintsugi, the practice of repairing broken pottery with liquid gold to make the finished piece stronger and more beautiful because it survived a fracture. In modern cloud architecture, systems break constantly. Traditional monitoring setups rely on static alert rules that page a human engineer when a threshold is crossed. This introduces an unacceptable delay in the Mean Time to Repair \(\text{MTTR}\), which directly degrades global system availability.

We engineered Kintsugi AI to rethink infrastructure resilience entirely. It is an autonomous, self-healing Site Reliability Engineering platform that acts as a continuous closed-loop defense layer. By introducing chaos engineering directly into the deployment loop, Kintsugi AI treats failures as runtime optimization steps, using artificial intelligence to instantly heal the fleet and secure transaction data.

The Engineering Mathematics Behind Kintsugi AI

To validate the platform efficiency, we can model system availability \(A\) through the traditional relationship between Mean Time Between Failures \(\text{MTBF}\) and Mean Time to Repair \(\text{MTTR}\):

$$A = \frac{\text{MTBF}}{\text{MTBF} + \text{MTTR}}$$

In standard operations, \(\text{MTTR}\) is a composite function of human variable friction:

$$\text{MTTR} = t_{\text{detection}} + t_{\text{triage}} + t_{\text{mitigation}} + t_{\text{verification}}$$

When an outage occurs, human engineers take minutes or hours to triage logs, causing the system to bleed through its strict Service Level Objective \(\text{SLO}\) error budget. Kintsugi AI compresses \(t_{\text{triage}}\) and \(t_{\text{mitigation}}\) down to seconds by evaluating raw telemetry arrays natively using a large language model.

The baseline anomaly detection threshold \(\tau\) for incoming latency metrics is dynamically calculated across our 25 regions using moving averages and a standard deviation sensitivity coefficient \(k\):

$$\tau = \mu + k \cdot \sigma$$

If the system latency metric climbs past the threshold (\(\text{p99} > \tau\)), our AI Sentry isolates the anomalous telemetry payload, triggers a recovery sequence, and ensures that the total consumed error budget is minimized over the incident duration:

$$\text{Error Budget Used} = \int_{t_0}^{t_1} \left( 1 - \text{Success Rate}(t) \right) dt$$

How We Built It

We designed Kintsugi AI as a responsive, real-time telemetry control room:

  1. The Core Framework: The frontend application layout is structured using Next.js and TypeScript, engineered for low-latency visual performance.
  2. The 3D Topology Grid: To map out our global fleet of 25 regions and 25,600 nodes, we built an interactive 3D WebGL globe using Three.js. Spherical mapping coordinates are mapped to the 3D space using standard trigonometric vector conversions: $$\begin{pmatrix} x \ y \ z \end{pmatrix} = \begin{pmatrix} -R \sin(\phi) \cos(\theta) \ R \cos(\phi) \ R \sin(\phi) \sin(\theta) \end{pmatrix}$$ Where \(\phi\) represents the latitude offset and \(\theta\) tracks the longitude.
  3. Live Streaming Data: Performance metrics are continuously fed into automated linear charts built on Chart.js that update every 900 milliseconds to reflect immediate structural changes.
  4. The AI Sentry Brain: Our orchestration engine routes raw incident context to the Gemini API. To ensure complete operational safety, we used Zod schema models to lock down the AI's response matrix. The model cannot hallucinate arbitrary commands: it must return structured object parameters selecting from a predefined list of valid mitigation scripts.
  5. The Persistent State Machine: Incident logs, root-cause analyses, and execution confidence percentages are committed immediately to an Amazon Aurora DSQL cluster, maintaining absolute distributed data integrity.

Challenges We Faced

Our biggest challenge was transforming a generative language model into a reliable, predictable system engineer. In early iterations, when a chaos test like a zone blackout was triggered, the model would occasionally output long descriptive paragraphs or suggest configuration actions that did not exist in our infrastructure limits.

We overcame this by creating a strict operational sandbox. By feeding the model strongly-typed runtime variables and enforcing strict structural schemas at the API gateway layer, we successfully eliminated non-deterministic behavior. If the AI analyzes an outage in a region outside of its primary network constraints, it safely falls back to guarded mitigation routines without crashing the system loop.

What We Learned & What is Next

Building Kintsugi AI taught us a massive amount about automated operations, telemetry pipelines, and token-based cloud database authentication. We learned how to integrate visual state engines like Three.js with deep asynchronous backend execution paths without dropping UI rendering frames.

Moving forward, we want to transition Kintsugi AI from simulated environments into real-world production setups. We plan to connect the Vercel orchestration backend directly to active AWS production node pools through secure runtime hooks. This will allow our AI Sentry to monitor actual live hardware, step in during genuine cloud infrastructure emergencies, and protect live software services around the clock.

Built With

Share this project:

Updates

Submission history