IT Ticket Classifier

AIpoweredIT helpdesk triage: anomaly detection+pattern learning autoresolve repeat tickets while genuine edge cases route to a human built on UiPath Maestro Case orchestrating a realFastAPI/ML backend

Comment

Inspiration

Most IT helpdesk tickets aren't new problems — they're the same handful of issues happening again and again. Yet most triage systems re-analyze every ticket from scratch, burning compute and time on patterns a human (or a model) has already solved a hundred times before. We wanted to build a triage system that actually learns — one that gets faster and more confident the more it sees a recurring issue, while still knowing when to step back and ask a human for help on something genuinely new.

What it does

The system takes an incoming IT ticket and routes it through a two-model pipeline:

  1. Anomaly Gate (Isolation Forest) — trained on 140k+ real ServiceNow tickets, scores each ticket on structured fields (reassignment count, urgency, impact, SLA status, etc.). Instead of a fixed percentile cutoff, it uses KDE-based valley-finding to set a threshold at the edge of the "normal" cluster, so it adapts to the actual shape of the data rather than an arbitrary rule of thumb.
  2. Pattern Store — every anomalous ticket gets compared against a running embedding-similarity store of known issue patterns. If it matches a pattern that's been resolved correctly enough times, the pattern graduates — and from then on, near-identical tickets get resolved instantly, skipping classification entirely.
  3. Classifier (KNN + conditional GMM) — anomalies that don't match a graduated pattern go to a KNN classifier (chosen over GMM-first after we empirically found GMM degraded on sparse classes), with GMM as a confidence cross-check using posterior probabilities, not raw likelihoods, to avoid biasing toward whichever class's distribution happens to be tightest.
  4. Human-in-the-loop — anything below a confidence threshold, or where KNN and GMM disagree, gets flagged for a human instead of auto-resolved.

The whole pipeline is orchestrated through UiPath Maestro Case, which treats each ticket as a case moving through stages — automated triage first, escalating to an explicit Human Review stage only when warranted — rather than a fixed flowchart that can't represent "this needs judgment."

How we built it

We started by training and comparing two architectures for the classifier stage before picking a winner with real evidence, not instinct: GMM-first didn't hold up once we tested it against the actual sparse-class distribution, so we made KNN primary and GMM a secondary cross-check. We then built the FastAPI backend stage by stage — anomaly detection, Pattern Store, classification, human-review branch — verifying each one against the real trained models rather than mocked stand-ins, and exposed it locally via ngrok so it could be reached as an external API.

The UiPath Maestro integration meant learning an entirely new orchestration paradigm: Maestro Case represents branching logic as declarative rules (entry/exit/completion conditions per stage) rather than drawn arrows, and an API Workflow task wraps the actual HTTP call to our backend, with nested If steps reading the live JSON response to decide whether a ticket needs human review.

Challenges we ran into

  • A double-increment bug in the Pattern Store — a single ticket was incrementing the pattern's occurrence count twice instead of once, because two different pipeline stages were both calling the same match-and-increment function on the same ticket. We caught it by noticing the count jumped 4→6 instead of 4→5, traced it to the exact call site, and fixed it by splitting out a read-only lookup function.
  • Silent label drift at graduation — we found that a pattern's centroid can drift toward a different issue category than its own stored label, purely through gradual fold-ins that each individually look fine. Since a graduated pattern skips the classifier forever, this could cause silent misrouting with nothing ever re-checking it. We added a label-consistency gate that re-verifies the classifier still agrees with a pattern's label at the exact moment it's about to graduate.
  • A one-character header bug that looked like a body bug — our first live UiPath→FastAPI call failed with a cryptic "JSON decode error," which looked like a malformed request body. The actual cause was a single leading space in the Content-Type header value, breaking FastAPI's body parsing entirely. Diagnosed by checking the raw request headers UiPath had actually sent, not just the payload.
  • Picking the right Maestro project type. Studio Web's main "Start modeling" button defaults into BPMN (fixed-flow) projects, not Case Management — easy to build the wrong artifact entirely for a system whose whole point is exception-driven branching. We caught this before building anything and switched to a real Maestro Case project.

What we learned

That trusting a model's first plausible-looking output is a habit worth resisting — both in our ML code (the label-drift gate exists because we asked "is this really working" instead of accepting a clean-looking graduation) and in the integration work (the header bug looked exactly like a body problem until we checked the actual bytes sent). We also learned that Maestro's rules-based approach to case branching — conditions on stage entry/exit rather than drawn connections — maps surprisingly well onto a system that's fundamentally a chain of if/else decisions in Python, once you find the right place to put each check.

What's next

  • Re-validate graduated patterns periodically, not just at the moment of graduation. Our label-consistency gate currently checks agreement once, right before a pattern graduates — but a graduated pattern could still drift afterward as more tickets fold in. A scheduled re-check (e.g. every N folds, or every retrain cycle) would catch drift that happens after the fact, not just before.
  • Smarter centroid updates. The current running-mean centroid treats every fold-in equally. An outlier-aware update — one that down-weights or rejects tickets whose individual classification strongly disagrees with the pattern — would make drift far less likely to begin with, rather than relying on a gate to catch it after.
  • Dynamic case input instead of a hardcoded test payload. Right now the Maestro case is wired to a fixed test ticket for verification. The natural next step is a real intake trigger — a webhook, a form, or a direct ServiceNow/email connector — so cases are created from genuine incoming tickets, not manually fired test runs.
  • A richer Human Review experience. Today, a flagged ticket waits for a human to resolve it through a basic endpoint. UiPath's Action Center / Action Apps could give reviewers a proper interface — ticket context, the model's reasoning, GMM/KNN disagreement details — so a human reviewer isn't resolving blind.
  • Deploy on UiPath Automation Cloud end-to-end, replacing the ngrok tunnel with a properly hosted backend (or a UiPath-native robot/connector calling the ML service), so the system doesn't depend on a laptop staying online.
  • Expand beyond IT helpdesk tickets. The anomaly-gate-then-classify-then-pattern-learn architecture isn't specific to IT support — the same shape could apply to insurance claims, customer support, or any high-volume ticketing domain where most incoming work is a repeat of something already solved.
  • Multi-pattern relationships. Right now each ticket matches at most one pattern. Real-world issues often have overlapping causes (a network issue causing a software crash) — modeling pattern relationships rather than treating each as independent could improve both classification accuracy and the quality of automated resolutions.
  • Feedback loop from human resolutions back into model retraining, not just into the Pattern Store — letting genuinely novel human-resolved cases improve the KNN/GMM classifier itself over time, not just feed pattern recognition.

Built With

Share this project:

Updates

Submission history