Factory AI Investigator

A maintenance copilot powered by NVIDIA Nemotron on Nebius that investigates anomaly alerts, gathers evidence through tools, and produces traceable reports with explicit uncertainty.

Comment

Inspiration

An anomaly alert tells a maintenance engineer that sensor behavior looks unusual, but leaves them to decide what evidence to inspect next.

We built Factory AI Investigator to connect sensor observations with operational records and technical documentation. Our goal is an assistant that investigates an alert, identifies missing evidence, and recognizes when it cannot justify a diagnosis.

What it does

The prototype supports five synthetic factory machines, M-101 through M-105. Users can inspect historical detection events, view sensor readings, request investigations, open cited evidence, and record shared review notes.

NVIDIA Nemotron on Nebius Token Factory selects evidence tools and reassesses their results. Tools compare sensor windows, inspect individual readings, retrieve earlier calibration records, search scoped machine logs, and retrieve relevant manual passages.

Each investigation preserves a fixed machine and timestamp scope. When evidence is insufficient, the report explains the remaining gaps and recommends additional evidence rather than inventing a root cause.

How we built it

A frozen XGBoost classifier detects unusual behaviour in 30-minute sensor windows using 96 engineered features.

Python computes sensor statistics and enforces tool arguments, timestamp boundaries, and execution limits. Nemotron selects validated observation IDs and evidence requests, then reassesses the evidence after tools execute.

Manual retrieval uses BGE embeddings and Qdrant. Reports include clickable evidence references, while saved traces record tool execution and reassessment decisions.

Google Cloud Run hosts the API and investigation jobs. BigQuery stores historical data and detection events, Cloud Storage holds artifacts and reports, and Firestore stores progress and shared reviews. Firebase Hosting and Authentication provide the dashboard and Google sign-in.

This edition builds on our earlier prototype. For this hackathon, we added Nemotron integration, iterative evidence seeking, structured selection validation, investigation traces, generalized five-machine tools, and a separate cloud deployment.

Challenges we ran into

Early model responses sometimes assumed sensor units, described endpoint differences as trends, or recommended repeating completed searches.

We addressed these issues by calculating and rendering numerical observations in Python, validating model-selected IDs, and excluding completed tool requests.

We also distinguished empty search results from tool failures. Missing records do not prove that nothing happened, and calibration history does not establish sensor health at the event time.

Cloud deployment required resolving service-account permissions, build configuration, authentication, and asynchronous progress updates.

Accomplishments that we're proud of

  • Generated cloud investigation reports and saved traces for all five machines.
  • Built a bounded investigation loop with visible tool execution and reassessment.
  • Connected sensor charts, reports, clickable evidence, and shared human reviews in one dashboard.
  • Independently verified one M-101 case against source CSVs, including sensor calculations, calibration filtering, log scope, and manual applicability.
  • Preserved uncertainty explicitly when the available evidence could not support a cause.

What we learned

Reliable agents need clear boundaries between model decisions and executable operations.

Nemotron decides which permitted evidence to request. Python performs calculations, enforces scope, validates selections, and records results.

We learned that retrieval similarity and valid citation IDs do not independently establish claim support. A useful investigation should show what was checked, what was found, and what remains unknown.

What's next for Factory AI Investigator

We plan to:

  • Compare the agent with a fixed retrieval workflow using a reproducible evaluation set.
  • Measure investigation latency, token usage, and evidence coverage.
  • Add tools for operating-state records and historical incidents.
  • Improve claim-level evidence validation.
  • Evaluate usability with maintenance engineers and verified equipment documentation.

The current prototype uses synthetic data and demonstration manuals. It supports human investigation; it does not confirm physical faults or replace approved maintenance procedures.

Built With

  • artifact-registry
  • bge-embeddings
  • bigquery
  • css
  • docker
  • fastapi
  • firebase-authentication
  • firebase-hosting
  • firestore
  • google-cloud
  • google-cloud-build
  • google-cloud-run
  • html
  • javascript
  • nebius-token-factory
  • numpy
  • nvidia-nemotron
  • pandas
  • python
  • qdrant
  • secret-manager
  • sentence-transformers
  • xgboost
Share this project:

Updates

Submission history