Pulse Oximeter Accuracy Dilemma: A Shift from Overestimation to Underestimation at Johns Hopkins

Pulse oximeters, widely popularized during the Covid-19 pandemic, have become essential tools for monitoring blood oxygen levels. However, they’ve recently come under scrutiny for their inconsistencies, notably their troubling tendency to inaccurately represent blood oxygen levels across different skin tones.

Research indicates that these devices often overestimate oxygen levels in patients with darker skin, potentially masking critical health issues and perpetuating inequalities in healthcare outcomes. A recent study published in the Annals of the American Thoracic Society expands upon these discrepancies. Researchers found a perplexing trend at Johns Hopkins Hospital, where pulse oximeters shifted from overestimating blood oxygen levels to underestimating them over a span of five years, with heightened inaccuracies seen in patients of color.

This shift raises significant questions. Ashraf Fawzy, a pulmonologist and one of the study's authors, remarked, “One of the biggest problems with pulse oximeters is that they are this black box. I have no idea why we found what we found in this paper, and why it’s changing year over year.” This sentiment echoes a sense of urgency around the need to understand the operational reliability of these devices.

In response to the findings, a spokesperson from Masimo, the company that manufactures the pulse oximeters used in the study, stated they were not aware of any changes in sensor technology, software, or manufacturing processes during the study period that could account for the observed discrepancies. They emphasized that multiple factors, including sensor types and placement, can influence the accuracy of readings.

Despite the uncertainties, researchers remain committed to understanding pulse oximeter performance. Fawzy's study utilized data from intensive care units at Johns Hopkins and noted that the readings from the same model of oximeter varied significantly over different evaluation periods. This inconsistency makes it challenging for clinicians to rely on readings, complicating timely and appropriate patient care.

A previous FDA-funded study, known as EquiOx, aimed to clarify the inaccuracies of pulse oximeters among patients with darker skin, only to reveal the unexpected trend of underestimation of blood oxygen levels instead of the anticipated overestimation. By comparing pulse oximeter readings against arterial blood gas tests—considered the gold standard—researchers confronted the complexities of interpreting results across varying presentations.

This new layer of uncertainty complicates clinical decision-making. If pulse oximeters only misrepresented oxygen levels in specific demographic groups, targeted adjustments could be made in care protocols. With devices showing inconsistent performance overall, this possibility becomes significantly more challenging. Thomas Valley, a health services researcher and pulmonologist, likened the situation to a broken clock, stating that if clinicians had insight into consistent inaccuracies, they could adapt their approaches. However, with current ambiguity about the nature of inaccuracies, it poses a considerable barrier to effective patient management.

Across the five-year analysis at Johns Hopkins, oximeters transitioned from overestimating blood oxygen levels—risking under-treatment of hypoxic patients—to underestimating them, potentially leading to unnecessary interventions or extended hospitalizations. Jack Iwashyna, a pulmonologist and co-author of the recent research, expressed concern about the utility of these devices, noting, “It is not at all clear these things are fit for use in the way that we’re using them.”

The ongoing discussion surrounding pulse oximeter performance continues to shed light on the necessity for rigorous evaluation of these tools. The landmark 2020 paper alerting the medical community to the devices’ racial bias relied on retrospective data, but prospective studies, aimed at investigating real-time usage, have yielded mixed results. Another recent study from Emory University confirmed racial bias but suggested that such inaccuracies could not be solely attributed to skin pigmentation, indicating a more intricate interplay of other health conditions influencing readings.

Sivasubramanium Bhavani, an ICU physician leading the Emory study, emphasized, “It is more complicated than just skin tone. We don’t know all the different causalities and how these things work.” Even with advances in calibration for skin tone, underlying health conditions prevalent in diverse populations could still skew pulse oximeter outputs.

As the medical community seeks clarity on pulse oximeter reliability, the findings from Johns Hopkins suggest an urgent need for transparency and improved calibration across diverse patient demographics. The lack of consistent, reliable data from these devices can have critical implications. Clinicians require tools they can trust when making vital decisions about patient care, and the push for better accuracy in pulse oximetry represents an important step toward more equitable healthcare outcomes.