OxiFair --- Skin-Tone-Aware Uncertainty Layer for Pulse Oximetry
Inspiration
Pulse oximeters are widely used to estimate blood oxygen saturation (SpO₂), but the reliability of a reading can vary with skin pigmentation and the specific device being used. OxiFair was inspired by the need to make this uncertainty visible rather than treating every SpO₂ value as equally reliable. The project focuses on a specific question: when should a clinician be less confident in a pulse-oximeter reading and consider confirmation or reassessment? The idea is grounded in published evidence showing differences in pulse-oximeter performance across skin pigmentation and device types. Rather than attempting to claim that every reading can be mathematically "corrected," OxiFair is designed to estimate uncertainty and communicate when additional confirmation may be appropriate. The central idea is: SpO₂ reading → uncertainty-aware prediction → confirm/reassess when warranted What it does OxiFair is a proposed software-only research and clinical decision-support prototype that adds an uncertainty layer around an existing pulse-oximeter reading. The system is designed to work with: SpO₂ measurement Pulse-oximeter device/model or device class Skin-tone information, such as Monk Skin Tone or objective Individual Typology Angle (ITA), where available Optional perfusion index Optional recent SpO₂ trend Instead of producing a single "corrected" SpO₂ value, OxiFair is designed to produce a prediction interval representing uncertainty around the estimated oxygen saturation. It can then provide a confirm/reassess flag when the lower bound of the interval crosses a clinically relevant threshold defined for the evaluation setting. OxiFair is intended to support clinicians and researchers. It is not designed to diagnose a patient or autonomously override clinical judgment. The core design principle is to quantify uncertainty rather than hide it.
How we built it
OxiFair was designed as a software-only research prototype, so the approach does not require modification of pulse-oximeter hardware.
- Defined the problem The first step was to focus on the documented relationship between pulse-oximeter performance, skin pigmentation and device characteristics. The project therefore treats skin tone and device identity as relevant sources of variation rather than assuming that one universal correction factor will work for every device and patient.
- Defined the required data The proposed modelling pipeline uses paired SpO₂ and SaO₂ measurements as the reference data. Where available, the dataset should also contain: Skin-tone measurements Pulse-oximeter device information Perfusion-related information Relevant recent measurements Objective skin-tone measurement such as ITA is preferred where a standardized measurement protocol is available.
- Modelled uncertainty The proposed modelling stage uses quantile or heteroscedastic regression to estimate the conditional behaviour of the reference SaO₂ value rather than relying only on a single point estimate.
- Calibrated the prediction intervals OxiFair then uses group-conditional conformal prediction to calibrate prediction intervals within relevant skin-tone and device groups. The objective is to evaluate whether the stated interval coverage is actually achieved across the groups being studied, rather than relying only on average performance.
- Added the decision layer The resulting prediction interval can be paired with a confirm/reassess flag. The intention is not to replace the original SpO₂ measurement with an invented number. Instead, the system communicates when the available evidence suggests that the measurement deserves additional confirmation or reassessment.
- Planned validation The primary evaluation focuses on: Prediction-interval coverage by subgroup Occult-hypoxaemia false-negative rate Differences in false-negative performance across skin-tone groups Comparison with unadjusted SpO₂ performance Average error alone is not treated as the sole evaluation criterion.
Challenges we ran into
Limited paired data A major challenge is the availability of paired SpO₂--SaO₂ measurements that also contain reliable skin-tone measurements and device identifiers. This limits how confidently a model can be calibrated for every possible skin-tone/device combination. Device-specific behaviour Pulse-oximeter performance can differ between devices. A universal correction factor could therefore be unsafe. OxiFair addresses this by treating device information as an important modelling and evaluation variable rather than assuming that one adjustment applies everywhere. Measuring skin tone Skin-tone measurement can itself introduce error. For this reason, the project prefers objective ITA measurements obtained using a standardized protocol where feasible. Avoiding false precision One of the biggest design challenges is deciding what to do when there is insufficient data for a particular subgroup. Instead of producing a seemingly precise correction from inadequate evidence, the proposed strategy is to abstain when the available data do not support reliable calibration. Clinical and regulatory considerations A system used for clinical decision support may have medical-device regulatory implications. OxiFair is therefore positioned as a research/decision-support prototype rather than a clinically deployed autonomous system.
Accomplishments we're proud of
The main accomplishment of OxiFair is turning the problem of skin-tone-related pulse-oximeter uncertainty into a concrete, testable software architecture. The project brings together: Pulse-oximetry performance analysis Skin-tone-aware evaluation Device-aware modelling Quantile or heteroscedastic regression Group-conditional conformal prediction Prediction intervals False-negative analysis A clinician-facing confirm/reassess concept Another important design decision was to avoid presenting a single "corrected" SpO₂ value as if it were universally accurate. Instead, OxiFair focuses on communicating uncertainty and measuring whether that uncertainty is appropriately calibrated across groups. The project is also deliberately designed to work as a software layer around existing pulse-oximetry infrastructure, which allows the research question to be investigated without first requiring new hardware.
What we learned
The project reinforced that average model performance is not enough when the goal is to evaluate a healthcare measurement across different groups. A model can appear accurate on average while still behaving differently for particular skin-tone or device groups. That makes subgroup-level evaluation essential. We also learned that uncertainty should be treated as an output rather than something hidden behind a single prediction. For OxiFair, this led to the decision to use prediction intervals and coverage metrics rather than simply producing a corrected SpO₂ number. Another important lesson is that device-specific behaviour matters. A model trained on one population or device configuration should not automatically be assumed to generalize to every pulse oximeter. Finally, the project highlighted the importance of knowing when not to make a prediction. If the available data cannot support reliable calibration for a subgroup/device combination, abstaining can be more appropriate than presenting false precision.
What's next
The next step is to implement and evaluate the research prototype using an appropriate paired SpO₂--SaO₂ dataset containing skin-tone and device information where available. The planned development path is: Identify and prepare suitable paired SpO₂--SaO₂ data. Standardize the available skin-tone and device variables. Establish an unadjusted SpO₂ baseline. Train the proposed uncertainty models. Apply group-conditional conformal calibration. Measure subgroup prediction-interval coverage. Measure occult-hypoxaemia false-negative rates. Compare performance across skin-tone and device groups. Evaluate whether the uncertainty layer reduces performance disparities without creating unsafe false reassurance. Build an auditable Streamlit research interface for displaying predictions, intervals and validation results. Longer term, the project could be evaluated across additional devices and datasets. Any future clinical deployment would require appropriate clinical validation, regulatory assessment and prospective evaluation rather than relying only on a research prototype. Will likely be Built With Python pandas NumPy scikit-learn Conformal prediction implementation
Research basis
The project is grounded in published and regulatory sources concerning pulse-oximeter performance and skin pigmentation, including: BMJ (2025) --- The impact of skin tone on performance of pulse oximeters used by NHS England COVID Oximetry @home scheme. The cited study analysed 11,018 paired SpO₂--SaO₂ measurements with objective skin-tone measurement. U.S. FDA (January 2025) --- Pulse Oximeters for Medical Purposes --- Non-Clinical and Clinical Performance Testing, Labeling, and Premarket Submission Recommendations. Draft guidance addressing performance considerations including skin pigmentation. U.S. FDA --- Pulse Oximeters. Information concerning pulse-oximeter accuracy limitations and skin-pigmentation considerations. PMC (2025) --- Pulse oximeter performance and skin pigment: comparison of 34 oximeters using current and emerging regulatory frameworks. The cited work reports device-specific differences in performance.
Important scope note
OxiFair is a proposed research/decision-support prototype. The project does not claim that it has clinically validated a universal correction for pulse-oximeter readings, nor that it can replace clinical assessment, confirmatory testing, better hardware, inclusive validation or regulatory oversight.
Log in or sign up for Devpost to join the conversation.