Triage IT AI

Every ticket is ready before a tech opens it. It reads every request the second it lands, works out what's wrong, and writes a finished ticket into your system. Your techs stop chasing details.

Comment

Inspiration

Good I.T. is a managed IT provider, so this problem came from our daily work. Support requests often arrive without the context a technician needs: no device name, no clear business impact, a deadline hidden in a paragraph, or an attachment that contains the most useful clue. Before the technical work can begin, someone has to identify the customer, reconstruct the history, inspect the affected device, search the knowledge base, check for related incidents, and decide what should happen next.

We did not want to solve that problem by placing an unsupervised chatbot in front of customers. A plausible but incorrect answer sent under an MSP's name creates real operational and reputational risk. We started with a different constraint: let AI do the repetitive reading, classification, retrieval, and drafting, while keeping a person responsible for anything the customer can see.

That constraint became Triage IT, an AI first line for IT service desks. It proposes, a technician confirms, and every decision is recorded.

What it does

Triage IT works alongside the service desk's existing tools and prepares each request before a technician begins the real work.

When a request arrives through Autotask, email, a customer portal, phone voicemail, or RMM telemetry, Triage IT can identify the client and contact, classify the request, assess urgency, extract deadlines, and identify missing information. Gemini can read screenshots to extract error codes, transcribe call recordings, and interpret attached documents. Triage IT enriches that request with device health from Datto RMM, relevant knowledge from an embedding-powered knowledge base, similar resolved tickets, and possible duplicate or incident relationships.

It then prepares the appropriate next step:

  • A draft answer for a simple, well-supported request.
  • A proposed customer reply for technician review.
  • A concise list of questions when important information is missing.
  • A structured handoff with history, risks, possible approaches, related tickets, and available scheduling windows when the issue needs a technician.

Customer-facing actions require human approval. Drafts appear as cards in Microsoft Teams, where a technician can approve, edit, or reject them. After approval, the card becomes a receipt showing who acted and when. Triage IT also creates draft knowledge articles from resolved tickets, quarantines likely junk for review, flags possible duplicates, and records a readable decision timeline for every ticket.

Triage IT is sold by Good I.T., its first production operator. It augments the PSA and collaboration tools the service desk already uses rather than asking the team to replace them.

How we built it

Triage IT is built with Python 3.11, FastAPI, PostgreSQL with pgvector, SQLAlchemy, Alembic, and Docker. The production console uses Microsoft Entra ID authentication. Integrations connect it to Autotask PSA, Datto RMM, Microsoft Teams, Microsoft Graph, email, and calendars.

Production AI requests run through Gemini on Vertex AI. We use gemini-2.5-flash for triage, drafting, handoffs, structured extraction, and multimodal understanding of images, audio, and documents. We use gemini-embedding-001 with pgvector for knowledge-base retrieval, duplicate and incident clustering, similar-ticket search, and learned junk detection.

The most important implementation decision was to use structured output for operational decisions. Gemini responses are bound to Pydantic schemas, so classifications, extracted fields, checklists, status choices, and routing recommendations must validate as typed data. A malformed decision fails explicitly instead of quietly entering the workflow as convincing prose.

Every external service sits behind an adapter with a working mock. A scenario simulator exercises the full pipeline without production credentials, and the automated test suite can run with the model faked. This allowed us to debate and regression-test behavior before connecting Triage IT to real client work.

We also built independent deployment gates for client scope, external writes, and shadow mode. In shadow mode, Triage IT processes real tickets and produces the decisions it would make, while its transport layer structurally blocks customer messages, PSA writes, and calendar changes. That let technicians evaluate the system before live actions were enabled.

Challenges we ran into

The first challenge was learning that a relevant knowledge-base article is not necessarily a useful answer. Early versions could find a topical article and treat that match as sufficient for self-service, even when the customer had already tried those steps. We added evidence and confidence thresholds, and weaker matches now become technician suggestions rather than customer answers.

The second challenge was separating actions that are usually correct from actions that are safe to automate. Duplicate detection is a good example: linking two related tickets is useful in most cases and damaging when the match is wrong. We converged on a practical rule: automatically apply only read-only or easily reversible actions, and ask a person before consequential changes.

The third challenge was integrating with the real dispatch process. Service-desk operations include ownership rules, SLA pressure, technician skills, workload balancing, scheduling, travel, and spillover between teams. Good classification alone is not enough. The recommendation has to fit the way work is actually assigned and completed.

Finally, measuring impact was harder than producing a compelling demo. Existing PSA timestamps were affected by automatic acknowledgements, so they could not support a valid response-time comparison. We chose to report only figures we could trace to an exact source and to document the limitations of each measurement.

Accomplishments that we're proud of

Triage IT progressed from an offline simulator, through shadow evaluation, to live operation on Good I.T.'s real Autotask workflow. In the measured window ending August 15, 2026, it processed 178 tickets from 15 client businesses and recorded 1,047 AI decisions across 25 decision types. That window includes both shadow and live operation, and we preserve that distinction in our reporting.

Technicians graded 38 AI-generated drafts during shadow mode: 21 would have been sent as written, 7 would have been edited, and 10 were rejected. That makes 73.7 percent usable, while still showing why human review matters. We are proud to report the rejected output alongside the successful output rather than hiding it.

Our most important accomplishment is the governance architecture around those decisions. Every ticket has a readable audit timeline. Customer-visible drafts wait for a person. Approval cards become buttonless receipts, and database locking prevents a message from being sent twice if multiple technicians click simultaneously. One configuration flag can return the entire system to shadow mode and structurally disable external actions.

We are also proud that Gemini is not a bolt-on chat feature. It participates throughout the ticket lifecycle: multimodal intake, triage, missing-information detection, knowledge retrieval, duplicate analysis, drafting, handoffs, status recommendations, knowledge capture, and dispatch briefings.

What we learned

Governance is not overhead added after the AI feature. It is the feature that makes operational AI usable. Trust grows when technicians can see a recommendation, disagree with it cheaply, and know that both the original decision and their response were recorded.

Structured output changes what can safely be built with a language model. Once a decision is a validated schema instead of prose, it can be tested, replayed, audited, constrained, and connected to deterministic business rules.

We also learned to deploy AI gradually. Running against real work in shadow mode exposed problems that synthetic demonstrations could not, while preventing those problems from reaching customers. Independent safety gates made each expansion of scope a deliberate operational decision.

Finally, we learned that measurement has to be designed into the product. Attractive numbers are easy to produce when definitions are vague. Useful evidence requires a known window, a traceable source, and an honest explanation of what the measurement cannot prove.

What's next for Triage IT

The next step is expanding from Good I.T., the first production operator, to other managed IT providers that use the same Autotask and Datto ecosystem. Each deployment will retain the staged client, write, and shadow-mode gates used for the first launch.

We will deepen dispatch support with technician skill tiers, pod and spillover rules, weighted workload scoring, travel-aware service calls, and technician-confirmed critical preemption. The goal is not merely to classify a ticket correctly, but to recommend an action that fits the real operating constraints of the service desk.

We also plan to connect proactive workflows directly to live RMM alerts, expand the technician feedback loop, improve multilingual support, and turn more resolved work into reviewed knowledge. Throughout that expansion, the product's core rule will remain unchanged: Triage IT can prepare and recommend the work, but a person remains accountable for consequential and customer-visible actions.

Built With

Share this project:

Updates