PriorPilot

Inspiration

Prior authorization is one of the most repetitive, time-consuming parts of getting patients care — providers have to gather records, review clinical history, and fill out insurer-specific paperwork by hand before treatment can even be approved. We wanted to see if an AI agent could take on that prep work end-to-end, without cutting corners on the parts that actually matter in healthcare: patient consent and secure data access.

What it does

PriorPilot automates the prep work behind a prior authorization request. It authenticates the user, gets explicit patient consent through a CIBA step-up flow, securely pulls clinical data (medications, conditions, labs) from Epic FHIR APIs via Auth0 Token Vault, and uses an AI agent to draft a structured PA form plus a clinical justification — grounded only in facts actually present in the records. The draft then goes to a simulated insurer portal for submission, but a clinician always reviews it first. PriorPilot drafts, it doesn't decide.

How we built it

React + TypeScript + Vite on the frontend, Node/Express on the backend. Auth0 handles authentication and brokers short-lived Epic tokens through Token Vault so we never store long-lived provider secrets. CIBA handles the patient consent flow. An agent loop built on OpenAI's function-calling takes the structured FHIR data and drafts the PA form + justification narrative. For submission, we built a real Playwright-driven headless-browser flow against a simulated insurer portal, instead of faking it with a timer.

Challenges We Ran Into

Agent Loop Function-Calling Sequencing — The trickiest part was managing OpenAI's legacy function-calling message flow once you're hand-assembling turns mid-stream. The SDK's ChatCompletionMessageParam union doesn't fully capture the shape of a function role message when you're building it yourself, so we had to use a looser type (AgentMessage) and let the SDK do the validation at send time. Even trickier: tool result messages MUST include the name field and MUST follow an assistant message with function_call, or the next API call gets a 400. A single misplaced message broke the whole loop.

Tool Result Capture Under Pressure — While iterating on the agent, we discovered we were calling tools but not always capturing their outputs into the session cache or extracting terminal artifacts (the form, the submission result). Lines 214–223 in agentLoop.service.ts are where we lock in the form and submission result as the agent produces them — if you miss assigning finalForm or submissionResult from a tool call, the caller gets nothing back, even though the agent saw it. We fixed this by explicitly checking fnName and casting the result.

JSON Parsing Failures in Function Arguments — OpenAI returns function arguments as a string; if the model hallucinates invalid JSON (rare but it happens), the agent loop would crash hard. We wrapped parsing in try-catch (lines 201–205) and log the error, but it still breaks the loop if not handled. We left the error logging in place so operators can debug.

Accomplishments We're Proud Of

Real-World Auth Flows, Not Mocked — The Token Vault + CIBA consent flow is ACTUALLY IMPLEMENTED. We don't fake patient consent or use a hardcoded token. Every time a clinician runs the workflow, the backend talks to Auth0, initiates CIBA step-up (patient gets a notification on their phone to approve), and then Auth0 Token Vault hands over a short-lived Epic access token. This is production-grade auth — no shortcuts.

Headless Browser Submission, Not a Timer — Most hackathon submissions fake the insurer portal with setTimeout. We built a real Playwright headless-browser flow (portalAutomation.Service.ts) that actually fills out the HTML form, clicks submit, and waits for the portal's confirmation element (lines 24–50). The portal responds with a reference number — we capture that, not a fake UUID. We could hand this off to a real insurer API tomorrow with minimal changes.

Clinical Data Grounding — The agent won't hallucinate diagnoses or medication names. Every claim in the justification comes from FHIR resources the agent actually fetched. The system prompt (lines 145–158) explicitly tells the model: "Never fabricate clinical data. If data is missing, note it clearly in the form." We validate completeness before submission (lines 325–332) and reject forms missing required fields.

End-to-End Pipeline That Actually Works — We took every part people usually mock (CIBA, Token Vault, headless browser, function calling) and made it real. A clinician can feed in a patient ID, hit "Process PA," and 60 seconds later, a form is drafted, validated, submitted to the portal, and a reference number is in hand. The whole thing works.

What We Learned

OpenAI Function-Calling Is Finicky At Scale — The model will sometimes ask for tools that don't exist, or call the same tool twice in a row, or return malformed JSON. We learned to validate every function_call before executing it and to have fallback responses. The agent loop's ability to continue reasoning after a tool result is its biggest strength, but you have to be religious about message sequencing, or it breaks silently in ways that are hard to debug.

Session State Is Fragile — We cache patient data, medications, and conditions in memory so the agent can reuse them across tool calls. But if the cache evicts or a call crashes mid-stream, the agent has to re-fetch. We learned: store session data in the session cache IMMEDIATELY after each tool fetch (lines 277–278, 284–285, 296–297), not at the end. Otherwise, you lose context on failure.

Patient Consent Is Non-Negotiable — CIBA flow adds 30–60 seconds because the patient has to approve on their phone. But we learned that cutting this step — even in a demo — trains bad habits. The system behaves differently when you actually wait for consent vs. when you fake it. We kept it in, even for hackathon judging, because trust is the whole point.