Inspiration
Software incidents can cause downtime, failed transactions, and frustrating user experiences. Engineers often need to examine application logs, metrics, and deployment events manually to determine what went wrong.
We wanted to explore how machine learning and AI agents could make incident investigation more systematic, evidence-driven, and efficient.
What it does
BLACK BOX AI is an AI-powered software incident investigation system designed to detect unusual application behavior, analyze operational evidence, investigate competing root-cause hypotheses, and recommend potential fixes.
Our planned workflow combines machine-learning-based anomaly detection with an NVIDIA AI model and diagnostic tools. Rather than relying on an AI-generated guess, the system aims to connect its conclusions to observable evidence and acknowledge uncertainty when the evidence is insufficient.
How we plan to build it
- Machine learning: Detect anomalous patterns in application metrics.
- AI investigation agent: Use an NVIDIA model through Nebius Token Factory to analyze evidence and coordinate diagnostic tools.
- Diagnostic engine: Examine application logs, performance metrics, and deployment events.
- Backend: Python and FastAPI.
- Frontend: React and TypeScript.
- Evaluation: Test against simulated incidents with known root causes and compare the results against simpler baseline approaches.
What makes it different
Our goal is to go beyond summarizing logs. BLACK BOX AI will investigate competing explanations, gather supporting evidence, and evaluate whether its diagnosis matches the known cause of a controlled incident.
Challenges we anticipate
We must prevent unsupported conclusions, distinguish correlation from causation, control false-positive anomaly alerts, and evaluate the reliability of the agent's recommendations.
We also aim to keep diagnostic actions controlled and require human approval before any remediation is applied.
What's next
We plan to implement the core investigation workflow, integrate an NVIDIA model through Nebius Token Factory, build an interactive dashboard, and evaluate the system using reproducible test scenarios.
Log in or sign up for Devpost to join the conversation.