Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learnedSmart Beneficiary Targeting Validation is a Databricks-native tool that flags CBPF (Country-Based Pooled Fund) humanitarian projects with abnormal cost-per-beneficiary ratios, benchmarked against comparable projects, and surfaces the comparison at proposal time rather than after funding is approved.

Built for UBC MSDS's Intro to Databricks hackathon (Challenge 1: Smart Beneficiary Targeting Validation), entered for the Best Use of Databricks prize.

The problem: CBPF fund managers approve hundreds of proposals per allocation round with no systematic check on whether a proposed beneficiary-to-budget ratio is normal for that type of project. The check today is manual, inconsistent, or skipped.

The approach: projects are grouped into peer groups by country, organization type, and allocation category before any cost comparison is made, so structurally different project types (e.g. logistics vs. WASH) are never compared against each other. Two independent, explainable signals are computed per project: a rule-based ratio to the peer group's median cost, and a baseline linear regression predicting expected budget. Both are surfaced side by side rather than collapsed into a single opaque score.

The build: an end-to-end Databricks pipeline, Bronze → Silver → Gold Delta tables on Unity Catalog, a baseline model tracked via MLflow, an AI/BI Genie space for natural-language queries over the results, and a Databricks App serving a live dashboard and Genie chat interface. Every component is native to Databricks.

Data: real CBPF project and contribution records, CERF allocations, HNO/HRP figures, and UN population data, sourced and verified from HDX and World Bank public datasets (see DATA_EXPLORATION.md).

Status: working baseline, not a final model. The regression is currently evaluated in-sample; the next iteration adds a held-out test set and a reviewer feedback loop so flag precision can be measured against confirmed outcomes.

What's next for DataBricks_hack

Built With

Share this project:

Updates

Submission history