PH Compass

PH Compass helps local health departments assess AI readiness, flag risks, and identify safeguards before adopting AI for public health tasks.

Comment

Inspiration

Federal public health agencies are rapidly scaling their use of AI. CDC reports that it has already generated 103 AI solutions or use cases and deployed a generative AI chatbot to all staff.\(^1\) At the same time, AI use cases across HHS agencies continue to grow, including reported increases at CDC, CMS, FDA, and NIH.\(^2\)

However local health departments are not always operating with the same resources, staffing, legal support, procurement systems, or technical infrastructure as federal agencies. In 2024, only 5% of local health departments in the United States reported using AI.\(^2\) Local public health agencies are still navigating major readiness barriers, including limited AI guidance, workforce skill gaps, data accuracy concerns, resource constraints, legal or policy barriers, and concerns about fairness and representativeness of data.\(^3\)

This gap inspired PH Compass.

We saw a clear need for a tool that helps local health departments move from broad AI curiosity to task-specific readiness. Instead of asking only, “Can AI do this?” PH Compass helps local health department leaders and staff ask, “Should we use AI for this public health task, right now, and what safeguards do we need first?”

We did not want to build another AI tool that simply tells public health agencies to adopt AI faster. We wanted to build a tool that helps local health department staff pause, ask the right questions, and move forward only when the task, data, people, safeguards, and workflow are ready.

References

  1. Rogers B. Considerations for GenAI Adoption. Office of Public Health Data, Surveillance, and Technology (OPHDST), Centers for Disease Control and Prevention (CDC). Presented June 5, 2026.
  2. Gleeson C. AI use is surging across HHS, jumping 148% at the FDA in 2025, Bipartisan Policy Center data finds. Fierce Healthcare. June 11, 2026. Accessed June 21, 2026. https://fiercehealthcare.com.
  3. The State of AI in Public Health: New Data from the 2025 ASTHO Profile. Association of State and Territorial Health Officials. April 28, 2026. Accessed June 21, 2026. https://astho.org.

What it does

PH Compass is an AI readiness assessment tool and navigator for local health departments. PH Compass helps local health department leaders and staff evaluate whether a proposed AI-supported public health task is ready to pilot safely such as drafting a public health advisory, translating emergency messages, summarizing community feedback, reviewing grant reports, supporting disease trend analysis, piloting a chatbot, or other AI use cases.

A local health department leader or staff member starts by entering details about the AI-supported task they are considering. PH Compass then guides users through seven readiness domains:

  1. Public Health Value
  2. Data Readiness
  3. Privacy & Legal Safety
  4. Equity & Community Impact
  5. Staff Capacity
  6. Governance & Oversight
  7. Technical & Pilot Feasibility

These domains reflect major themes in public health AI guidance: mission alignment, human oversight, accuracy, scientific integrity, transparency, privacy, secure tools, training, monitoring, equity, and governance.

PH Compass generates a readiness score using a transparent weighted rubric:

$$\text{Readiness Score} = \sum_{i=1}^{7} w_i \times s_i$$

where \(w_i\) is the domain weight and \(s_i\) is the task-specific score for that domain.

The score is not the whole decision. PH Compass also applies risk penalties and red-flag overrides. For example, a task may score well overall but still receive a Do Not Use Yet recommendation if it involves protected health information in an unapproved tool, public-facing health guidance without review, eligibility or enforcement decisions without human oversight, or translated emergency messages without fluent human review. This matters because generative AI outputs can be inaccurate or fabricated, and high-risk uses require oversight. It also matters because information entered into generative AI may create legal, privacy, or public records concerns.

The final output gives the user:

  • An AI readiness score out of 100 points. Public Health Value is weighted at 20 points; Data Readiness, Privacy & Legal Safety, Equity & Community Impact, and Staff Capacity are each weighted at 15 points; and Governance & Oversight and Technical & Pilot Feasibility are each weighted at 10 points.
  • An AI readiness category, such as Ready to Pilot, Pilot with Guardrails, Pause and Prepare, or Do Not Use Yet
  • Domain-by-domain strengths and gaps
  • Risk flags
  • Recommended safeguards
  • Staff support needs
  • Recommended next step
  • A plain-language explanation for leadership, IT, legal, communications, and program staff

How we built it

PH COMPASS uses a single-page React client and a stateless ASP.NET Core API. The API is split into an application layer, deterministic policy modules, and provider-specific infrastructure. Assessments are evaluated in memory and are not persisted. A more in depth description of the system design, source code, and models is provided at this link.

Backend Design

The backend follows a layered design:

  • Controllers handle HTTP routing, request limits, cancellation, and response contracts.
  • Application services coordinate classification and deterministic evaluation through interfaces.
  • Assessment modules collect domain signals, apply focused policy rules, calculate scores, normalize findings, and build the report.
  • Scoring policy centralizes weights and provisional penalty values.
  • Infrastructure owns Gemini, CORS, rate limiting, health checks, deployment behavior, and Problem Details.

Frontend Design

The frontend is feature-oriented rather than page-oriented:

  • App.tsx owns request cancellation, submission state, API errors, and the transition between questionnaire and report.
  • AssessmentForm.tsx owns the current step, draft data, step validation, and final request construction.
  • steps/ contains focused views for task, people, impact, data, readiness, and review.
  • FormControls.tsx supplies shared accessible radio, checkbox, heading, and “Other” detail controls.
  • assessments.ts is the only browser API boundary.
  • AssessmentResults.tsx renders score arithmetic, classification, domain evidence, policy gates, safeguards, navigation, and technical metadata.
  • types/assessment.ts mirrors the backend wire contract.

AI Analysis Process

The AI portion is intentionally narrow. It interprets task language but cannot score readiness or choose a recommendation. The deterministic analysis stages are:

  1. Validate questionnaire and classifier output.
  2. Reconcile task labels with explicit AI-use selections.
  3. Convert answers and risk tags into 0–5 domain signals.
  4. Average and weight the seven readiness domains.
  5. Apply centrally configured numeric penalties.
  6. Apply mandatory Pause and Prepare orDo Not Use Yet gates.
  7. Deduplicate and prioritize findings.
  8. Return score arithmetic, evidence, safeguards, staff support, and a safer alternative when appropriate.

Coding References

Frontend

Backend

AI and Deployment

AI Usage

The coding used is mostly implemented or similar to walkthrough tutorials coding snippets. The use of AI while coding was minimal, such as function/step auto-completion. AI agents were used for testing the frontend of the deployed build, this was used to automate a systematic testing on our framework for scoring the readiness of an AI-assisted tool development for public health departments.

Challenges we ran into

Among coding challenges, such as rewriting parts of the code that end up not adhering to modularity, many times throughout the testing we faced issues with the formula itself. The formula is not perfect which leads to another issue: no feedback loop with users is possible at this stage. In terms of testing, we also faced issues when dealing with nonsense input. In the first iteration of the formula, although we could select unsure for some of the prompts, the app and formula didn't handle well random inputs such as writing/selecting "unsure" for every single text box. Some rewriting of the AI scoring system had to be done, and there are many more uncovered issues that we could benefit learning from an AI vulnerability assessment.

Accomplishments that we're proud of

We are proud that PH Compass turns a complex AI governance problem into a usable public health decision-support tool for local health department staff.

Instead of telling local health departments to either adopt AI or avoid it, PH Compass gives a more realistic answer: some AI-supported tasks are low-risk and useful, some need training or policy review first, and some are too risky without stronger safeguards.

We are also proud that the tool treats equity and public trust as core readiness issues, not optional add-ons. PH Compass asks whether an AI-supported task could worsen disparities, exclude communities, reduce language access, affect services, or create unintended harms.

We are proud that PH Compass is task-specific. A department can compare multiple possible AI uses using the same readiness structure while still recognizing that each task has a different risk profile.

Most importantly, we are proud that PH Compass keeps humans in control. The tool is not designed to replace public health judgment. It is designed to help local health department staff make better, safer, and more accountable decisions about AI.

What we learned

We learned that AI readiness in public health is not just about whether a tool works. It is about whether the department has the right task, data, staff capacity, review process, legal safeguards, and community trust to use that tool responsibly.

One of our biggest takeaways was that the same AI capability can be low-risk or high-risk depending on the task. Using AI to draft internal grant language is very different from using AI to translate an emergency alert, support inspection follow-up, or power a public-facing chatbot. Because of that, AI readiness has to be assessed by use case, not just by department.

We also learned that readiness is not permanent. A department may not be ready today, but with training, approved tools, legal review, staff roles, pilot testing, and evaluation, it may become ready later. PH Compass is designed to show that path.

What's next for PH Compass

Some of the expected future versions of PH Compass is to include the following: 1.Implement authentication and authorization, basic security principles to any multiuser system.

  1. Session management.
  2. An enforced security assessment of the platform.
  3. Communication between team members, such as approving tool ideas.
  4. A multi-agent loop system for updating task ideas with new information.
  5. A feedback system for updating the formula used for calculating the AI readiness.

Built With

Share this project:

Updates