Humanitarian Project Budget Prediction & Explainability

Overview

This project uses Machine Learning and Generative AI to analyze humanitarian funding allocations and provide explainable budget predictions for proposed humanitarian projects.

Built as part of a Databricks hackathon, the system leverages historical humanitarian funding data from the UN OCHA Pooled Funds API, supplemented with humanitarian needs data (HNO), to predict an appropriate funding amount for a given project based on its characteristics.

Beyond generating predictions, the system uses SHAP (SHapley Additive exPlanations) to identify the factors influencing each prediction and an LLM to translate these contributions into clear, human-readable explanations.

The goal is to support transparent, data-driven funding analysis and help identify projects whose requested or approved budgets differ significantly from historical patterns.

Problem Statement

Humanitarian organizations allocate funding across projects that differ in geographical location, sector, target population, duration, and severity of humanitarian needs.

Evaluating whether a proposed budget is consistent with comparable historical projects can be challenging.

Our solution addresses this by:

  1. Predicting project budgets: Train an ML regression model on historical humanitarian project data to estimate funding amounts.
  2. Identifying budget deviations: Compare predicted budgets with requested or approved amounts to flag unusually high or low allocations.
  3. Explaining predictions: Calculate local SHAP values to identify which project characteristics drive individual predictions.
  4. Generating actionable insights: Use an LLM to convert model predictions and SHAP contributions into understandable explanations.

Data Sources

  • UN OCHA Pooled Funds API: Historical project funding, approved budgets, project characteristics, sectors, locations, and implementation details.
  • Humanitarian Needs Overview (HNO): Contextual information about humanitarian needs, including affected populations and severity indicators where available.

Solution Architecture

UN OCHA Pooled Funds API    HNO Data
            |                  |
            +--------+---------+
                     |
               Data Ingestion
                     |
          Databricks Delta Tables
                     |
         Cleaning & Feature Engineering
                     |
           ML Regression Model
                     |
              New Project Input
                     |
              Budget Prediction
                     |
             Local SHAP Analysis
                     |
              LLM Explanation
                     |
           Prediction + Insights

ML Inference and Explainability

For each new humanitarian project, the system follows this workflow:

  1. Accept project characteristics as input.
  2. Preprocess the inputs using the same transformations applied during training.
  3. Generate a predicted budget using the trained regression model.
  4. Calculate local SHAP values to quantify each feature's contribution to the prediction.
  5. Pass the prediction, baseline, and most influential feature contributions to an LLM.
  6. Generate a natural-language explanation highlighting factors increasing or decreasing the predicted budget.
  7. Return the predicted amount, budget deviation (if a requested or approved amount is provided), and explanation.

Databricks Integration

The solution is designed to leverage the Databricks ecosystem:

  • Delta Lake: Store and manage ingested humanitarian project data.
  • Databricks Notebooks: Data exploration, preprocessing, feature engineering, and model development.
  • MLflow: Track experiments, evaluate models, and manage model versions.
  • Databricks Model Serving: Deploy the trained model for inference.
  • Databricks Foundation Model APIs: Generate natural-language explanations using an LLM.
  • Unity Catalog: Govern datasets and model assets.

Expected Output

For a new humanitarian project, the system can return:

  • Predicted budget: $245,000
  • Requested budget: $300,000
  • Deviation: +22.4% relative to the prediction
  • Key influencing factors: Target beneficiaries, humanitarian severity, project duration, and historical funding patterns.

Example explanation:

"The model estimates a budget of $245,000 for this project. The number of targeted beneficiaries and severity of humanitarian needs contribute positively to the prediction, while historical funding patterns contribute negatively. The requested budget is approximately 22% higher than the model's estimate, suggesting that the allocation may warrant further review."

These values are illustrative. Predictions represent historical funding patterns rather than definitive assessments of appropriate humanitarian spending.

Project Goal

Our objective is not to replace human decision-making in humanitarian funding but to provide an explainable, data-driven decision-support tool that improves transparency, highlights unusual funding patterns, and helps stakeholders make more informed budget assessments.

Built With

Share this project:

Updates

Submission history