Black Box Whisperer

Making legacy systems readable, explainable, and safe without touching their code.

Comment

About the Project

Inspiration

This project started from a recurring and uncomfortable observation we both shared: many of the most critical systems in production still work perfectly, yet no one truly understands them anymore.

In large organizations, legacy systems are often:

  • business-critical
  • technically stable
  • undocumented or poorly documented
  • understood by only one or two people (sometimes no longer there)

The real risk is not that these systems fail, but that we cannot explain what they do, why they behave this way, or what would happen if something changes.

Black Box Whisperer was born from this gap between operational reliability and human understanding.

What We Learned

Building this project together reinforced several key insights:

  • Understanding a system is not the same as reading its code.
  • Deterministic analysis is essential before any AI reasoning.
  • AI becomes truly useful only when it works on structured, verifiable evidence, not raw data.
  • Legacy systems require respect: rewriting them is often more dangerous than understanding them first.

Technically, we deepened our understanding of:

  • static analysis pipelines
  • graph-based representations of software behavior
  • clear separation between deterministic computation and probabilistic reasoning
  • designing AI systems that remain explainable and auditable

How We Built It

The project is structured around a strict separation of concerns.

1. Deterministic Analysis Engine

The core engine performs non-intrusive analysis:

  • parsing legacy source code (COBOL in V1)
  • extracting symbols, calls, and dependencies
  • correlating logs and observable artifacts
  • building call graphs and dependency graphs
  • computing risk indicators and metrics

This layer is fully deterministic and reproducible. No AI is involved at this stage.

Formally, the engine produces a set of structured artifacts:

[ Artifacts = {Graphs, Metrics, Evidence, Normalized\ Events} ]

These artifacts act as the ground truth.

2. AI Reasoning Layer

The AI layer operates only on the structured outputs of the engine.

Its role is to:

  • reconstruct business logic in human language
  • generate hypotheses with explicit evidence references
  • reason about failure scenarios and cascades
  • produce operational insights such as runbooks and risk explanations

The AI never analyzes raw code or logs directly. This design choice keeps reasoning explainable and falsifiable.

3. User Interface

A desktop UI allows users to:

  • load and validate analysis runs
  • explore artifacts and graphs
  • inspect integrity and mismatches
  • navigate explanations and supporting evidence

The UI is intentionally passive: it visualizes and explores results, but does not perform analysis itself.

Challenges We Faced

The hardest challenges were not raw technical complexity, but design discipline.

  • Resisting the temptation to let AI analyze everything
  • Defining what deterministic understanding really means
  • Designing abstractions that can scale to multiple legacy languages
  • Keeping the system explainable at every step
  • Managing scope and focus within hackathon time constraints

We constantly had to balance ambition with realism: building a system that is genuinely useful, not just impressive.

Conclusion

Black Box Whisperer does not aim to replace engineers. It aims to give them back a clear mental model of systems that have become opaque over time.

We do not rewrite legacy systems. We understand them. And understanding is the first step to making them safe.

Built With

Share this project:

Updates