• 

• 

OxiFair - Skin-Tone-Aware Uncertainty Layer for Pulse Oximetry

Inspiration

The idea for OxiFair began with a personal experience. In January 2026, while caring for my grandmother after her heart bypass surgery, I spent time around hospital monitoring equipment, including a pulse oximeter.

During one conversation, a senior resident doctor jokingly remarked that “the oximeter is racist.” It was a light-hearted comment, but it made me curious about the serious technical issue behind it: can the reliability of a pulse-oximeter reading vary depending on a patient's skin pigmentation or the device being used?

That question stayed with me and honestly after going through tons of ideas i decided to give this a go and found a very solid backing and research grounded evidence.

Pulse oximeters are widely used to estimate blood oxygen saturation (SpO₂), but published research has reported differences in their performance across skin pigmentation and between devices. This raised a more specific question for me:

When should a clinician be less confident in a pulse-oximeter reading and consider confirmation or reassessment?

I did not want to approach this by simply creating a universal mathematical "correction" for SpO₂. A correction could give the impression that an uncertain measurement has become perfectly accurate.

Instead, OxiFair was built around a different idea: make uncertainty visible.

The central concept became:

SpO₂ reading → uncertainty-aware prediction → confirm/reassess when warranted

This led to OxiFair, a proposed skin-tone- and device-aware uncertainty layer for pulse oximetry.

---

What it does

OxiFair is a proposed software-only research and clinical decision-support prototype designed to add an uncertainty layer around an existing pulse-oximeter reading, without requiring modification of the pulse-oximeter hardware.

The system is designed to consider:

SpO₂ measurement

Pulse-oximeter device/model or device class

Skin-tone information, such as Monk Skin Tone or objective Individual Typology Angle (ITA), where available

Optional perfusion index

Optional recent SpO₂ trend

Rather than outputting a single supposedly "corrected" SpO₂ value, OxiFair is designed to estimate a prediction interval representing uncertainty around the estimated oxygen saturation.

The proposed pipeline is:

Input → Model → Calibration → Decision → Validation

The modelling stage can use quantile or heteroscedastic regression, followed by group-conditional conformal prediction to calibrate prediction intervals within relevant skin-tone and device groups.

The resulting output is intended to contain:

Estimated oxygen saturation + prediction interval + confirm/reassess signal

For example, if the lower bound of the prediction interval crosses a clinically relevant threshold defined for the evaluation setting, OxiFair can flag the reading for confirmation or reassessment rather than presenting the original SpO₂ value with unwarranted confidence.

An important part of the design is that OxiFair does not assume that one correction factor should work across every patient and pulse oximeter. Device-dependent behaviour and limited data for some skin-tone/device combinations are explicitly treated as limitations.

Where the available evidence is insufficient to support reliable calibration for a particular group or device, the proposed approach is to abstain rather than fabricate precision.

OxiFair is intended to support clinicians and researchers. It is a research/decision-support prototype and is not intended to diagnose patients, autonomously make clinical decisions, or override clinical judgment.

The core principle behind the project is:

• Don't hide uncertainty behind a single number. Quantify it, communicate it, and make it measurable.

How we built it

OxiFair was designed as a software-only research prototype, so the approach does not require modification of pulse-oximeter hardware.

1. Defined the problem The first step was to focus on the documented relationship between pulse-oximeter performance, skin pigmentation and device characteristics. The project therefore treats skin tone and device identity as relevant sources of variation rather than assuming that one universal correction factor will work for every device and patient.
2. Defined the required data The proposed modelling pipeline uses paired SpO₂ and SaO₂ measurements as the reference data. Where available, the dataset should also contain: Skin-tone measurements Pulse-oximeter device information Perfusion-related information Relevant recent measurements Objective skin-tone measurement such as ITA is preferred where a standardized measurement protocol is available.
3. Modelled uncertainty The proposed modelling stage uses quantile or heteroscedastic regression to estimate the conditional behaviour of the reference SaO₂ value rather than relying only on a single point estimate.
4. Calibrated the prediction intervals OxiFair then uses group-conditional conformal prediction to calibrate prediction intervals within relevant skin-tone and device groups. The objective is to evaluate whether the stated interval coverage is actually achieved across the groups being studied, rather than relying only on average performance.
5. Added the decision layer The resulting prediction interval can be paired with a confirm/reassess flag. The intention is not to replace the original SpO₂ measurement with an invented number. Instead, the system communicates when the available evidence suggests that the measurement deserves additional confirmation or reassessment.
6. Planned validation The primary evaluation focuses on: Prediction-interval coverage by subgroup Occult-hypoxaemia false-negative rate Differences in false-negative performance across skin-tone groups Comparison with unadjusted SpO₂ performance Average error alone is not treated as the sole evaluation criterion.

Challenges we ran into

Limited paired data A major challenge is the availability of paired SpO₂--SaO₂ measurements that also contain reliable skin-tone measurements and device identifiers. This limits how confidently a model can be calibrated for every possible skin-tone/device combination. Device-specific behaviour Pulse-oximeter performance can differ between devices. A universal correction factor could therefore be unsafe. OxiFair addresses this by treating device information as an important modelling and evaluation variable rather than assuming that one adjustment applies everywhere. Measuring skin tone Skin-tone measurement can itself introduce error. For this reason, the project prefers objective ITA measurements obtained using a standardized protocol where feasible. Avoiding false precision One of the biggest design challenges is deciding what to do when there is insufficient data for a particular subgroup. Instead of producing a seemingly precise correction from inadequate evidence, the proposed strategy is to abstain when the available data do not support reliable calibration. Clinical and regulatory considerations A system used for clinical decision support may have medical-device regulatory implications. OxiFair is therefore positioned as a research/decision-support prototype rather than a clinically deployed autonomous system.

Accomplishments we're proud of

The main accomplishment of OxiFair is turning the problem of skin-tone-related pulse-oximeter uncertainty into a concrete, testable software architecture. The project brings together: Pulse-oximetry performance analysis Skin-tone-aware evaluation Device-aware modelling Quantile or heteroscedastic regression Group-conditional conformal prediction Prediction intervals False-negative analysis A clinician-facing confirm/reassess concept Another important design decision was to avoid presenting a single "corrected" SpO₂ value as if it were universally accurate. Instead, OxiFair focuses on communicating uncertainty and measuring whether that uncertainty is appropriately calibrated across groups. The project is also deliberately designed to work as a software layer around existing pulse-oximetry infrastructure, which allows the research question to be investigated without first requiring new hardware.

What we learned

The project reinforced that average model performance is not enough when the goal is to evaluate a healthcare measurement across different groups. A model can appear accurate on average while still behaving differently for particular skin-tone or device groups. That makes subgroup-level evaluation essential. We also learned that uncertainty should be treated as an output rather than something hidden behind a single prediction. For OxiFair, this led to the decision to use prediction intervals and coverage metrics rather than simply producing a corrected SpO₂ number. Another important lesson is that device-specific behaviour matters. A model trained on one population or device configuration should not automatically be assumed to generalize to every pulse oximeter. Finally, the project highlighted the importance of knowing when not to make a prediction. If the available data cannot support reliable calibration for a subgroup/device combination, abstaining can be more appropriate than presenting false precision. What's next The next step is to implement and evaluate the research prototype using an appropriate paired SpO₂--SaO₂ dataset containing skin-tone and device information where available. The planned development path is: Identify and prepare suitable paired SpO₂--SaO₂ data. Standardize the available skin-tone and device variables. Establish an unadjusted SpO₂ baseline. Train the proposed uncertainty models. Apply group-conditional conformal calibration. Measure subgroup prediction-interval coverage. Measure occult-hypoxaemia false-negative rates. Compare performance across skin-tone and device groups. Evaluate whether the uncertainty layer reduces performance disparities without creating unsafe false reassurance.

Research basis

The project is grounded in published and regulatory sources concerning pulse-oximeter performance and skin pigmentation, including: BMJ (2025) --- The impact of skin tone on performance of pulse oximeters used by NHS England COVID Oximetry @home scheme. The cited study analysed 11,018 paired SpO₂--SaO₂ measurements with objective skin-tone measurement. U.S. FDA (January 2025) --- Pulse Oximeters for Medical Purposes --- Non-Clinical and Clinical Performance Testing, Labeling, and Premarket Submission Recommendations. Draft guidance addressing performance considerations including skin pigmentation. U.S. FDA --- Pulse Oximeters. Information concerning pulse-oximeter accuracy limitations and skin-pigmentation considerations. PMC (2025) --- Pulse oximeter performance and skin pigment: comparison of 34 oximeters using current and emerging regulatory frameworks. The cited work reports device-specific differences in performance. Important scope note OxiFair is a proposed research/decision-support prototype. The project does not claim that it has clinically validated a universal correction for pulse-oximeter readings, nor that it can replace clinical assessment, confirmatory testing, better hardware, inclusive validation or regulatory oversight.

Built With

• python