Privacy Lab

PrivacyLab: what does protecting customer privacy cost a bank, and when is sharing data worth it?
Real fraud models trained on 284,807 credit card transactions. At strong privacy (ε = 0.5), fraud losses rise from $2,653 to $6,738.
The privacy level from the Lab sets the game. At strong privacy, pooling is worth it, but only with a partner you trust.
Each quarter, share your fraud data or keep it private. Both moves are revealed at once, along with what each bank saved.
Sharing every quarter with a fair rival saved $22,200, which is $9,200 more than never sharing.
Team PrivacyLab at DubHacks 2026.

We connected machine learning, business costs, and game theory in one working experience.

Inspiration

Privacy has a price. But how do you put a number on it?

Organizations want to catch fraud without exposing the people behind their data. Protecting privacy can affect model performance, while combining data with another organization introduces another question: can you trust your partner to cooperate?

We built PrivacyLab to make these trade-offs understandable through real experiments and an interactive game.

What it does

Experiment Lab: Compare fraud detection models across privacy levels. See how much fraud they catch, how many false alarms they generate, and what those mistakes cost. Adjust the cost of reviewing a false alarm to explore different business assumptions.

We turn model mistakes into a simple business cost:

$$ \text{Modeled Cost} = \text{Missed Fraud Amount} + (\text{False Alarms} \times \text{Cost per Review}) $$

Strategy Room: Bring your selected privacy level into a six-quarter banking simulation. Choose whether to share data or keep it private, respond to a rival’s hidden behavior, and review the consequences.

What we discovered

Combining data barely helped one model and transformed another.

At our strongest privacy setting, fraud detection rose from 14.6% to 52.5%, lowering modeled cost by about $2,732. Without privacy protection, combining data saved just $10.

These results come from two simulated banks drawn from one dataset.

What we learned

Better models don’t automatically make better partnerships. Combining data can help, but partners still need a reason to share. That gap between technical benefit and human cooperation became the heart of our game.

Challenges we ran into

Spotting rare fraud: A model could label everything legitimate and still look accurate. We needed measures that exposed its mistakes.
Preserving useful patterns: We had to prepare bounded training inputs without losing the signals that help detect fraud.
Making research playable: We connected real experiment results to a game while clearly labeling its assumed rewards and penalties.

How we built it

We built the machine learning pipeline with Python, scikit-learn, and diffprivlib, using a real credit card fraud dataset. We evaluated private models across five random seeds to capture variation.

The pipeline exports results to a React and TypeScript application, with Recharts visualizations and deployment on Vercel. Training runs offline; users explore the results in their browser.

The privacy guarantee covers model training, not the entire application or validation process.