FirstNotice Dispatch

Upload accident evidence, not forms. FirstNotice Dispatch uses Gemini-powered agents to build claims, catch bad evidence, coordinate next steps, and escalate when human judgment is needed.

Comment

Inspiration

After an accident, people are already stressed, yet insurance intake often asks them to manually re-enter information that already exists in their photos, documents, and even in the story they can simply tell.

I wanted to reduce that burden.

FirstNotice Dispatch is built around a simple idea: upload evidence, not forms.

Instead of asking a claimant to reconstruct an accident field by field, FirstNotice uses the evidence they already have — vehicle photos, policy documents, reports, and voice responses — to build and move the claim forward.

The goal is not to automate every insurance decision. It is to automate repetitive coordination while keeping humans in control when judgment matters.

What it does

FirstNotice Dispatch is an AI-powered auto insurance claim intake and orchestration system.

A claimant uploads accident evidence, and Gemini-powered agents can:

  • extract structured claim details from documents, images, and voice
  • determine whether enough evidence exists to continue
  • detect missing, unusable, or conflicting evidence
  • request only the specific information needed to unblock the claim
  • validate corrective evidence when the claimant responds
  • resume a paused workflow automatically when the issue is resolved
  • stop autonomous processing when human judgment is required
  • coordinate downstream actions such as Gmail notifications and inspection scheduling

One of the key behaviors is evidence remediation.

For example, if FirstNotice cannot verify the insured vehicle, it pauses the claim and asks for a clearer vehicle photo. New evidence is validated against the grounded claim context before the workflow is allowed to continue.

If incident details are still missing, the claimant can respond naturally by voice instead of filling out another form. Gemini extracts grounded facts such as when the accident happened, what happened, and whether an injury was mentioned.

That new information can also change the workflow.

If the claimant mentions an injury, FirstNotice does not continue autonomously just because the missing fields are now complete.

FirstNotice does not just know when to act — it also knows when to stop.

The workflow pauses and routes the completed evidence package to an adjuster for human judgment.

After the adjuster authorizes a physical inspection, the same durable workflow resumes automatically, sends the final handoff, and creates the inspection in Google Calendar.

How I built it

I designed FirstNotice Dispatch as an event-driven agent workflow on Google Cloud.

The claimant experience is built with Angular, while backend services run on Cloud Run.

Gemini 3.5 Flash provides multimodal reasoning over images, PDFs, and claimant voice, with Google ADK used for agent orchestration.

The system uses:

  • Gemini 3.5 Flash for multimodal evidence understanding and reasoning
  • Google ADK for agent orchestration
  • Cloud Run for the web application, claimant API, and private dispatcher
  • Pub/Sub for asynchronous workflow events
  • Firestore for durable claim and workflow state
  • Cloud Storage for uploaded image, PDF, and audio evidence
  • Gmail for adjuster and claimant notifications
  • Google Calendar for inspection coordination
  • Angular and TypeScript for the frontend
  • Python, FastAPI, and Pydantic for backend services and structured contracts

I intentionally separated AI reasoning from consequential workflow control.

Gemini handles ambiguous multimodal understanding and structured evidence analysis, while deterministic application logic owns state transitions, idempotency, evidence requirements, routing, and safety boundaries.

That allows the system to be agentic without letting model output alone make consequential insurance decisions.

Challenges I ran into

One of the hardest problems was deciding when new evidence should be considered useful, conflicting, or simply unusable.

For example, a replacement vehicle image may contain a matching license plate while other grounded facts such as make or model conflict with the insured vehicle. The workflow cannot rely on one matching field and assume the evidence is valid.

The system first determines whether the evidence is usable, then compares the available grounded facts before deciding whether the outstanding requirement is actually resolved.

Another challenge was making pause-and-resume behavior reliable.

Corrective evidence has to stay associated with the exact outstanding request so FirstNotice can distinguish original evidence from remediation evidence and safely resume the same claim rather than starting over.

Voice remediation introduced a similar challenge. A natural voice response needed to resolve missing incident details while preserving provenance and allowing the new information to alter the workflow when necessary.

The final challenge was defining the boundary between autonomous action and human judgment.

I did not want the agent to automatically advance sensitive claims simply because enough structured information had been collected. Human-review states therefore became first-class workflow outcomes rather than exceptions.

Accomplishments that I'm proud of

I’m proud that FirstNotice Dispatch became much more than a document-extraction demo.

It can reason over evidence across multiple steps, pause when information is missing or unreliable, validate corrective evidence, accept natural voice responses, and resume the same claim automatically when the problem is resolved.

I’m especially proud of the human-in-the-loop boundary.

When the claimant supplies new information that changes the risk of the workflow — such as mentioning an injury — FirstNotice stops autonomous routing and brings in an adjuster.

The adjuster receives the relevant evidence package and claimant context through a secure review flow, and approval resumes the same workflow rather than starting a separate process.

The system then takes real external actions by sending Gmail notifications and creating the physical inspection in Google Calendar.

I’m also proud of the separation between probabilistic reasoning and deterministic workflow control. Gemini handles ambiguous multimodal understanding, while application logic owns consequential transitions, idempotency, routing, and safety policies.

What I learned

The biggest lesson was that useful agents are not just about generating answers.

For this workflow, the system has to reason continuously about:

What do I know? What is missing? Can I trust this new evidence? Does it contradict what I already know? Can I continue safely, or should I stop?

I also learned that probabilistic AI reasoning and deterministic workflow controls complement each other.

Gemini is strong at understanding ambiguous multimodal evidence.

Deterministic systems are better suited for state transitions, idempotency, safety policies, and consequential actions.

Most importantly, I learned that good agent design includes knowing when not to act.

Human escalation is not a failure of the agent. In the right situation, stopping and asking for judgment is exactly the correct autonomous behavior.

What's next for FirstNotice Dispatch

FirstNotice Dispatch could expand beyond first notice of loss into a broader evidence-driven claims workflow.

Future directions include:

  • additional evidence types and richer multimodal input
  • stronger provenance and fraud signals
  • configurable insurer-specific workflows
  • adjuster feedback loops
  • richer inspection and repair coordination
  • additional human-review policies
  • improved claimant status communication
  • more advanced evidence-conflict resolution

The broader vision is to reduce repetitive work for claimants and adjusters while allowing agents to safely coordinate the work that happens in between.

Built With

Share this project:

Updates