FaultTrace

Inspiration

A plant alarm tells you something is wrong. It rarely tells you what evidence to collect next—or whether a completed repair actually restored operation.

FaultTrace explores that gap: helping a remote investigator and an on-site operator work through an incident together, with AI proposing observations, people making decisions, and software independently checking recovery.

What it does

FaultTrace provides two connected workspaces:

• Commander: reviews telemetry, field evidence, observation proposals, correction approvals and recovery progress.
• Operator: receives equipment assignments, submits photographs and structured observations, reports inaccessible equipment and acknowledges approved simulated work.

Our demonstration uses a simulated cooling return line with a pump, filter and valve. The investigation preserves competing explanations while collecting evidence. A simulated correction requires explicit human approval against the current evidence revision. Completion alone does not establish recovery: a separate verifier checks consecutive post-action telemetry samples and revokes recovery if conditions deteriorate.

How we built it

The application uses React and TypeScript with a FastAPI backend, SQLite persistence and private image storage.

NVIDIA Nemotron models run through Nebius Token Factory. Nemotron Nano proposes observations from a bounded action catalog. Nemotron Super VL interprets photographs into structured observations, including visible readings, units and uncertainty.

Application policy validates model proposals before creating tasks. Evidence references, revisions, idempotency and persisted receipts keep the investigation traceable. Simulator fault labels remain outside model context.

Challenges and lessons

Structured output can be valid while its interpretation is wrong. Our first completed real-photograph analysis returned a contract-valid gauge reading that disagreed with preliminary human review. We preserve that result as a failure requiring evaluation.

Working with Nebius Support also helped us confirm the hosted vision model’s reasoning controls and resolve output truncation.

These experiences reinforced our central design choice: model claims, human observations, correction authorization and recovery verification must remain distinct.

Accomplishments

We built a complete offline investigation-to-recovery journey, shared Commander and Operator records, persistent evidence and approval history, and automated checks covering stale evidence, disputes, refusal, interruption, ineffective correction and relapse.

What’s next

Complete live unified-workflow evaluation, desktop and mobile browser acceptance, server-enforced access controls and deployment. Evaluate photographic accuracy using independently reviewed images before making broader reliability claims.

FaultTrace currently demonstrates simulated equipment workflows; it does not control real equipment.

Built With

• fastapi
• github-action
• nebius-token-factory
• nvidia-nemotron
• pillow
• python
• react
• sqlite
• typescript
• vite