WORK AS DONE AGENT

An AI agent that turns frontline adaptations into evidence for human-reviewed improvement—without judging workers or rewriting standards.

Comment

Inspiration

Written procedures describe how work is expected to happen. Real work, however, often succeeds because people make small, careful adaptations when conditions change.

These adaptations are rarely recorded. They may remain with one experienced worker, disappear at the end of a shift, or be dismissed as simple deviations. I wanted to explore whether AI could help preserve this practical knowledge without monitoring, ranking, or judging workers.

WORK AS DONE AGENT was inspired by frontline experience and by a human-first principle: AI can organize evidence, but people must decide what becomes shared knowledge or a candidate for changing a standard.

What it does

WORK AS DONE AGENT captures a short account of what a worker did slightly differently, why it was necessary, the operating conditions, and which essential checks were preserved.

The Agent then:

  • compares the observation with the written standard;
  • searches previous observations for similar adaptations;
  • compares causes and operating conditions rather than wording alone;
  • uses NVIDIA Nemotron to reason across the current case and relevant history;
  • builds a Practice Variant Map showing different methods and their shared core;
  • separates what is known, unknown, and still needs evidence;
  • creates a human-review candidate when a pattern begins to repeat.

The AI does not rank workers, approve a method, or rewrite the SOP. It stops at the review boundary. A person can retain the observation, request more evidence, share a provisional field note, create a standard-review candidate, or reject the pattern.

A reviewed field note can also be linked to a later bounded trial. This allows the team to record what happened next while keeping the written standard unchanged.

How we built it

The application uses a FastAPI backend with a browser-based interface designed around a five-screen evidence flow:

  1. Today
  2. Record actual work
  3. Agent investigation
  4. Pattern evidence
  5. Human review

NVIDIA Nemotron-3-Nano-30B-A3B runs through Nebius Token Factory. The model compares the written standard, current operating context, preserved checks, and relevant historical observations.

The Agent’s visible activity log shows each step, including saving the observation, loading the standard, searching history, comparing operating conditions, running Nemotron reasoning, building the Practice Variant Map, and creating a review candidate.

All workplace records shown in the demo are synthetic.

Challenges we ran into

The main challenge was not simply connecting a model. It was defining where the model should stop.

A system that automatically labels a workaround as good or bad could overlook safety, local rules, physical load, or context that was never recorded. A system that silently rewrites a standard would also remove responsibility from the people who understand the work.

We therefore designed explicit evidence boundaries. Safety, quality, time, physical load, and rules remain visible as known, unknown, or needing evidence. The interface also states clearly that an AI finding is not an approved work instruction.

Another challenge was distinguishing genuinely similar adaptations from observations that merely use similar words. The Agent compares causes and operating conditions so that evidence is grouped by context rather than surface language alone.

Accomplishments that we're proud of

We completed a working end-to-end prototype with live NVIDIA Nemotron reasoning through Nebius Token Factory.

The prototype can:

  • capture an anonymous frontline observation;
  • compare it with a written standard;
  • identify recurring adaptations across historical records;
  • preserve multiple practice variants without declaring one worker correct;
  • expose missing evidence and unresolved risks;
  • stop before making a policy decision;
  • record a human decision;
  • link a later trial result while leaving the SOP unchanged.

I am especially proud that the product demonstrates AI as an additional participant in workplace learning—not as a supervisor replacing human judgment.

What we learned

The most valuable frontline knowledge is often not a universal best practice. It is conditional knowledge: what worked, under which conditions, for what reason, and with which essential checks preserved.

We also learned that uncertainty should be part of the product, not hidden from the user. Showing missing safety evidence or unknown physical-load effects makes the Agent more useful and more honest.

AI becomes more trustworthy when its authority is deliberately limited. The point is not to automate the final decision. The point is to give people better evidence for making it.

What's next for WORK AS DONE AGENT

Next, we want to test the workflow with more synthetic scenarios and then conduct a small, carefully bounded pilot with frontline participants.

Future development will focus on:

  • multilingual observation capture;
  • stronger privacy and access controls;
  • document-based ingestion of workplace standards;
  • evidence quality and recurrence thresholds;
  • longitudinal comparison of reviewed field notes and trial results;
  • clearer reporting for safety, quality, time, and physical-load review.

The long-term goal is to help organizations learn from work as it is actually done while keeping human responsibility, local knowledge, and worker dignity at the center.

Built With

Share this project:

Updates

Submission history