Pulse oximeters give biased results for people with darker skin

https://bostonreview.net/science-nature-race/amy-moran-thomas-how-popular-medical-device-encodes-racial-bias

by lilrhody • 6 years ago

180 111 6 years ago

bostonreview.net

vr464• 6 years ago

Quelle surprise - garbage in, garbage out. I'll bet my Apple Watch doesn't have these issues.

ChrisLomont• 6 years ago

I'll bet my Apple Watch doesn't have these issues.

I bet they do. They certainly did a while back, and I've found nothing saying this has been fixed.

https://bossip.com/1137485/race-matters-new-apple-watch-malf...

vr464• 6 years ago

Mildly interesting, but that looks exceedingly anecdotal.

bildung• 6 years ago

Apple Watch already had exactly this problem in the past.

vinay4272• 6 years ago

The Apple Watch doesn't have a pulse oximeter (EDIT: it does have one that is not approved and therefore is disabled), so it would be difficult for it to have this issue. You might be thinking of optical HR measurement, for which sensors have also been shown to have skin color biases in some cases.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7010823/

ceejayoz2• 6 years ago

The Apple Watch doesn't have a pulse oximeter

Yes, it does. The FDA blocked its use.

https://www.city-journal.org/fda-blocks-apple-watch-blood-ox...

https://www.cultofmac.com/320322/apple-watch-sensors-are-cap...

vr464• 6 years ago

Ah yeah, maybe. I've had the watch and previously a Vivoactive HR and they never seemed to have any trouble with my skin colour when measured against other measurement methods.

PaulHoule• 6 years ago

If I understand it, they looked at 3 pulse ox meters. The expensive one from the company that was sponsoring the study read about one point (out of 100) lower on blacks which is inaccurate but not outrageous.

One of the off-brand meters was completely wrong (e.g. dangerous) for blacks, the other one was in between.

simion3142• 6 years ago

They look at a few studies too and asked around companies if they handled the issues mentioned in those studies, the most positive result was that some models from one company are now better)probably some of the others are not).

IMO seems that the certifications for this devices, especially for the ones used by medical stuff to decide things need to be better tested.

Other interesting point that I did not consider is you can have a device with an average error under a safe threshold but that gives a much above average error for some groups putting this groups at risk if the people using this devices are not warned about this issues.

IndPhysiker• 6 years ago

Nonin created the first of these back in the '90s, so I pulled one of their spec sheets from 2016 (https://www.nonin.com/wp-content/uploads/2018/09/NoninConnec...) which explicitly mentions the pigmentation issue along with nail polish, poor circulation, and breathing issues. From that, this isn't exactly a new or unknown thing since commercial products have been working around this for some time. I wasn't able to find an industry standard for testing on these, so maybe the article should cite that as an opportunity to improve a product or add a new method?

bluntfang• 6 years ago

I don't think this should be flagged/removed. It's a technical review of medical devices.

shalmanese• 6 years ago

I wonder if the pulse oxes in China are calibrated for Chinese skin or White skin? Many people are buying pulse oxes straight off of Alibaba and they might be getting readings that are too low if the calibration is off.

dogma1138• 6 years ago

I’ll be surprised if they are calibrated at all.

jschwartzi• 6 years ago

Lots of thermometers on Amazon aren’t even FDA cleared so I wouldn’t be surprised if the pulse oximeters aren’t FDA cleared either.

oneplane6• 6 years ago

Depends on what you are measuring as well; if you want to know if there is a significant change you can do that. If you want absolute numbers you are probably doing it wrong anyway. (there is no single true template for numbers universally describing what a certain human should be like)

CWuestefeld• 6 years ago

China is home to a tremendous amount of racial diversity, from the majority Han to Tibetans to the oppressed Uighur. Our western idea that "Asian" is a race is rather flawed. I don't think you can come up with any "typical" Chinese skin color.

FWIW, my wife (Han, who grew up in Shanghai) generally has whiter skin than do I (a generic American of mostly northern European descent).

pkaye2• 6 years ago

I saw a video somewhere of a pulse ox which would given fake reasonable reading if you put a pencil in its sensor. Just makes you think its working.

treis3• 6 years ago

This really isn't a big deal. Doctors know that these pulse ox meters aren't very accurate and use them accordingly. Nobody's changing their plan because you have a pulse ox of 93 instead of 95. They're there to quickly measure big changes that indicate a problem. In other words, they're looking for patients going from 98 to 75, not 98 to 95.

jrajav• 6 years ago

What's the point of casually dismissing this article's premise? Whether the conclusions are fully sound or not isn't the goal of this article, this is journalistic not academic. The goal is to raise awareness of a potential bias, and I think it achieves that (assuming the facts all check out).

For one, according to the article the discrepancy was found to be up to 8 points in some cases, not 3. And it seems reasonable that quick, practical decisions in hospitals are made based on point thresholds, not purely huge point leaps as you suggest.

I haven't dug any further than the article and your comment to verify these facts. But rather than find a way to prove why this doesn't matter, why don't we assume it does? Then the knowledge and awareness might spread a little bit more through us, and if it is an issue, it might be that much more likely to be solved. There's no penalty for being wrong with a weakly held but well-meaning assumption. At the very least, we can be more aware of another possible dimension of bias in technology.

x86_64Ubuntu• 6 years ago

This thread because it has race as a secondary theme will be rife with such statements dismissing the premise.

treis3• 6 years ago

What's the point of casually dismissing this article's premise?

Because it should be casually dismissed. This person doesn't know what they're talking about, refused to listen to the people who told them they didn't know what they are talking about, and wrote an article full of nonsense.

prvc• 6 years ago

why don't we assume it does? Then the knowledge and awareness might spread a little bit more

If the assumption is incorrect, then it would be misinformation that we would be spreading.

jennyyang• 6 years ago

Medicine is very rarely about absolutes, it's mostly about how things get better or worse. When I was in the hospital, no one looked at the absolute values of my blood work, they cared to see if it dropped based on the treatment they saw.

Also, doctors are already aware about how oximeters may be inaccurate. It doesn't mean that they still can't be useful.

btach• 6 years ago

Many quick, practical decisions are made on patient assessment and clinical judgement, not just numbers. None of us that I know of has ever come across a perfect pulse ox. "Treat the patient, not the numbers" as the saying goes. When they don't match up, I like to do things like use a different finger/hand (or ear lobe or forehead).

diydsp• 6 years ago

FTA: "At a reading of 88 or 89, Medicare will reimburse for oxygen at home, but at 90 it won’t"

tssva• 6 years ago

Getting oxygen at home reimbursed by Medicare also requires diagnoses with certain conditions and specific testing. If oxygen at home is required the testing to satisfy Medicare requirements won't be conducted using the common finger tip pulse ox meter.

openasocket• 6 years ago

There might actually be room for a lawsuit there, if that regulation has a demonstrable impact on people with darker skin

in_cahoots• 6 years ago

That hasn’t been true at all in my experience. When my son got pneumonia, we were told he would be hospitalized at 92 when he was reading 94. He’s dark-skinned so who knows what the actual reading should have been.

tashi• 6 years ago

My insurance company didn't authorize a full sleep study because my overnight pulse oximeter results were normal, so it took me an extra couple of years to find out I needed a CPAP machine. It might have helped to know that my skin color could cause inaccurate results.

dragonwriter• 6 years ago

Nobody's changing their plan because you have a pulse ox of 93 instead of 95.

Threshold values are used to.make decisions, dark skin people do read higher (meaning decisions not informed by knowledge of the difference are likely to delay intervention for darker skinned people) and also have worse outcome with otherwise similar presentation for many of the conditions where pulse oximetry would be a factor in decision-making. It's quite possible that the outcome difference is in part because of the measurement difference, even if the likely impact in each individual case is minor.

rscho4• 6 years ago

We are looking for people to make 90+, 92+ if reasonably healthy. 75 is super low, and apart from exceptions is a _very_ concerning value. See the dissociation curve of hemoglobin/O2 to know why.

However yes, in my experience I never had enough difference in dark-skinned individuals to be clinically meaningful.

zamfi2• 6 years ago

Frustratingly, this kind of dismissal perpetuates injustices.

First, let's be clear:

Doctors know that these pulse ox meters aren't very accurate and use them accordingly

Some doctors know...

Nobody's changing their plan because you have a pulse ox of 93 instead of 95. They're there to quickly measure big changes that indicate a problem.

Sometimes, perhaps even often, they're used for this. It's one use case of pulse oximetry, and it's an important one.

I read your post as extrapolating general claims from common cases, and erasing the real experiences of people that fall outside of these common cases.

Bias often happens at the margins. That doesn't make it less real for the people who suffer from it, even if most people don't suffer, even if the common cases work well.

It can be hard to see those margins when you aren't in them, but that doesn't mean they don't exist -- they are still worth writing about and trying to fix.

CositaS• 6 years ago

Also, while they might not change plan because of a reading of 93 instead of 95 - the article describes a bias of up to 8%, which does seem enough to be significant.

slowmovintarget• 6 years ago

This IS NOT BIAS! This is physics!

As someone else pointed out, this isn't only about melanin in the skin. Darkened skin for any reason will cause this (burns, smoking, dirt, etc...). Medical personnel are trained to account for this partial accuracy.

The other fact is that there are newer techniques being worked on to improve the technology.

Teknoman117• 6 years ago

I'd hardly call it injustice. Anything based on light is going to be variable depending on skin tone (or, as other have mentioned, darkened skin for any reason).

The only thing you could claim injustice for would be if there was no other mechanism for measuring blood oxygen other than something that was known to not work properly for people of color.

sangnoir3• 6 years ago

In other words, they're looking for patients going from 98 to 75, not 98 to 95

The problem is that that '95' might really be a '90' - surely a drop from 98 to 90 is a cause for concern. If that information is being hidden, it needs to be fixed, regardless of other, more subjective fallback methods.

JamesBarney• 6 years ago

The difference of 5% is only common in the 60-70% on some clips. In the 90%-100% level the different is up to 2%-3% on the worst pulse ox's, and usually around 1%.

For reference I have two pulse oximeters at home and they routinely give me results that differ by 2-3%.

Seems like an issue we should fix, but I don't know what impact this would have in a clinical setting given that they use multiple criteria and the devices are known to be inaccurate.

ceejayoz2• 6 years ago

Reminds me of the automatic soap dispenser at Facebook that can't see black people. https://gizmodo.com/why-cant-this-soap-dispenser-identify-da...

ravenstine2• 6 years ago

I can believe this because automatic doors seem more prone to having a hard time opening when I wear black clothes. It's not a big deal or anything, but I occasionally have to step back and then forth again to get it to open. Is this a common experience of black people, or am I crazy?

exikyut2• 6 years ago

Oh, THAT'S why entrance barriers at supermarkets only occasionally fail, but fail consistently on a given day...

Wait. Would this happen to PIR-based systems?

PascLeRasc3• 6 years ago

I'm a student EMT and our instructor told us not to rely on a pulse ox reading if our patient has dark skin, or if they were in a fire, or are a lifelong smoker, or if they have any nail polish or dirt/grease on their fingertips. We just look for the 94-100% range and would only take action if we get a reading around 87% or lower, or see a sudden drop during treatment.

There are a few alternatives designs that aim to mitigate this particular issue by either preventing crosstalk [1] or by using multiple LED wavelengths and angles - the second one is something a med device engineer told me about a few months ago, but I'm having trouble finding any papers or information on it.

[1] https://www.hindawi.com/journals/jhe/2018/3521738/

Edmond• 6 years ago

The article notes that the devices inaccurately report higher oxygen levels for darker skin tones. Meaning if someone dark skinned reports a result that falls into the 94-100% range, they may in fact be at lower oxygen levels that put them at risk.

PascLeRasc3• 6 years ago

That's true. There are other context clues that we can use, like a patient's medical history, how their breathing looks/sounds, or if their environment or mental status suggests CO poisoning (that also won't show up even on the perfect pulse ox, since it binds to hemoglobin the same way oxygen does). Overall we just need a better way to get information about oxygen levels.

treis3• 6 years ago

The bias is smaller than the actual measurement error. The measurement error is around +/- 1. So a 96 reading means actual blood O2 of 95-97. The bias introduced by skin color is around 0.5. So a 96 reading means 95-97 in a white person and 94.5-96.5 in a black person. The bias from skin color isn't significant because the measurement itself is pretty inaccurate.

m463• 6 years ago

here is one:

https://web.archive.org/web/20200708064647/https://www.howeq...

rscho4• 6 years ago

Personally, I've never seen a case where the difference was clinically meaningful. So, meh...

RcouF1uZ4gsC• 6 years ago

Yeah, and if there was a real pressing need to get an more accurate number, you would get an arterial blood gas.

Pulse oximetry is for monitoring change. When a critical value is needed for medical intervention, such as a decision to put in a breathing tube, tests like arterial blood gas has are used.

tomphoolery• 6 years ago

This device seems pretty flawed if it can't read fingers on like 80% of people...

rscho4• 6 years ago

If people were in a fire it's true the value may be flawed, but we usually say that more because of potential carboxy- and methemoglobinemia than because of soot.

adrianmonk2• 6 years ago

We just look for the 94-100% range and would only take action if we get a reading around 87% or lower

I'm not very familiar, so I'm not sure how to interpret this. Does this describe the normal procedure for when you can rely on a pulse oximeter, or does it describe the modified procedure for when you can't?

PascLeRasc3• 6 years ago

That's the normal procedure, as far as I know. Taking a pulse ox reading is done at the same time as we'd get blood pressure, listen to breath sounds and watch chest expansion depth/rate, and talk to the patient about how they're feeling or what's wrong. It's just one factor that could tell us that they're short of breath and we should provide oxygen, but it's important to consider the patient as a whole. They like to say "treat the patient, not the data".

godelski• 6 years ago

Just so you have some extra information (I actually build a heart rate monitor and pulse oximiter) skin color and transparency makes a big difference, same with the color of the LEDs. I'm pretty pale and one of my friends is pretty tan, my signal was much stronger than his. The IR (infrared) ones work better as they are able to penetrate the skin better and there are clearer absorption lines from oxidized blood. I also tested different locations to place the device, I did notice most are fingertip or wrist (fitbits and smart watches), but found I got substantially stronger signals by placing the device near the upper forearm (god did this help in debugging). This was true for everyone I tested (unfortunately only white people) and helped smooth out the data a little, but it was still noticeable.

On how these devices work, it is typically two LEDs the produce light that bounces through the skin, some light is absorbed by blood (different absorption by different oxidation and density) then the reflected light is captured by a photodiode. All of this is on one surface, they are very basic. The density changes rapidly and that's how you capture the pulse (easy to get btw) and the oxidation is based on absolute levels (THIS IS THE PROBLEM!!!!). After building one I learned to not take these things too seriously, especially fingertip and wrist mounted ones.

I encourage others to build them. You can do most of it in a weekend if you have some soldering experience (best if you can do surface mount soldering), some programming, and are comfortable doing data analysis. And no, I do not expect my device to be on par with the ones in hospitals, but I was quickly able to match readings from a finger based device that I bought from the local drug store.

oneplane6• 6 years ago

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved.

I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result.

There are very real problems of course, take the example with the photographic film development from the article: that is a problem because a picture that is supposed to reflect the subject you photographed no longer does what it was suppsoed to do. And someone here posted the example of the soap dispenser that couldn't "see" people that didn't have light skin. Also a problem, because now you can't get soap. Both of those are mostly examples of errors made during engineering (apply Hanlon's razor when in doubt - it's not always malice like people often assume), and might be preventable if there were more different samples and interactions available during development.

That doesn't mean it's the only type of problem we have with devices for humans, or that keeping a color palette of people around is always going to fix it.

Take the soap dispenser example: you could have made one that uses radar or ultrasonic detection, solves multiple problems because you now don't have to consider lighting conditions which always vary due to differences in bathrooms (light fixtures, positioning, positioning of the dispenser itself etc). That doesn't relate to skin color, but can just as easily be a similar problem that could have been prevented all the same by doing proper engineering. But it's likely that in cases like this (soap dispensers) the detection system was just a module that some other third party company imports and sells, and they bought it from a module integrator somewhere else and they bought the parts as generic motion detectors from a company 4 layers down the chain that specializes in measuring daytime movements of rabbits or something silly like that.

On the other hand: if you have a government that requires certain controls to be in place and applied for medical equipment to be used in a medical setting you would expect some of the requirements to be drawn up in a way that reflects the use cases. It appears that either the devices are not scoped within those controls, or the controls failed. Or it's all good and the difference in measurement is not significant enough to matter for the use case and thus the devices are used as-is and this research doesn't relate to the real world as well as people think.

function_seven2• 6 years ago

Both of those are mostly examples of errors made during engineering (apply Hanlon's razor when in doubt - it's not always malice like people often assume), and might be preventable if there were more different samples and interactions available during development.

When I see these types of examples brought up, I don't see malice being assumed. Rather, these are good illustrations of structural bias. "Structural" (or "systemic") is often used in contrast to overt bigotry or malice. It's sometimes the result of a certain workforce or industry being substantially skewed with respect to the wider population. You're right, Hanlon and his razor do work here to show that it's a naive error rather than intentional harm. But the result is still traceable to the disparities between the designers and the population at large.

As for the soap dispenser: If the team of engineers designing the module had a mix of skin colors, then they would have quickly seen that their product failed sometimes. They would, at the outset, have known that they needed to tune the photoreceptor better, or switch to a different method of detection.

Though it is funny that Nikon, a Japanese company, made cameras with an blink detector that falsely triggered on Asian eyes. That either refutes my own point above, or it indicates they imported that functionality.

oneplane6• 6 years ago

The issue is that we assume that there was a building somewhere, where a group of people sat down and designed and produced a device. (the soap dispenser) But we don't actually know that. If there is a single spot where this was done, and also where the testing was done, and there was an abundance of skin tones available, then this was caught. But that's a lot of ifs.

It could be that the device as designed and completed for manufacturing 15 years ago, the manufacturing was shifted around between 2 companies and the supply chain has 10 levels deep of participants that all did their own tests on their part in their own company in their own country on their own continent. And between that and the consumer it turned out there were 4 companies that just trade and don't actually look at, touch or use the product.

This is probably not what happened at Nikon, I'd guess that they have a few departments that may not work together as well as they could. Would be interesting to see how this actually came to be, I imagine there are many ways to create a (partially (bad)) product, and equally many ways to try to prevent that.

tolbish• 6 years ago

Just because your customer didn't explicitly tell you to account for people with darker skin doesn't mean your failure to do so is just "an error made during engineering". It can still be systemic racial bias for neither you nor your client to consider who is actually going to be using your devices.

Also, no one said this is the only problem we have with devices.

vmception2• 6 years ago

The product just wouldn't have shipped with a product manager who also experienced it not working.

Thats the point of inclusion.

Although the soap dispenser example feels like one of those problems, it is just wear and tear of the sensor and they work properly when cleaned.

In this case, pulseox failures, can have effects on when people are admitted for treatment.

oneplane6• 6 years ago

It would probably not have shipped in that case, but the amount of product managers actually using their own products sadly isn't at the saturation you'd expect.

Same with all the white labeling and supply chain and integrators in between; the disconnect between all of the stages and the contracts that can bind a bad product to be delivered and sold anyway can't be fixed on just one end of the process.

rscho4• 6 years ago

The product just wouldn't have shipped with a product manager who also experienced it not working.

Yes sure, why don't we dispense with our most important and practical monitoring system because it has a racial bias? Pulsoxymetry saves lives every day. Many lives. Just ask old caregivers who were around before them.

jfengel2• 6 years ago

Each instance of it isn't necessarily malicious, but the way they all end up causing harm to the same people makes the system as a whole act as if it were malicious. Especially when combined with a proportion of people who are actively malicious, and when any attempts to address the systemic issue are dismissed.

When it comes to us in particular, we can each truthfully say "it wasn't my fault and I didn't mean any harm". But harm happens, and Hanlon's razor becomes dull with repeated application like that. As another rule goes, "Once is happenstance, twice is coincidence, three times is enemy action".

The solutions to systemic problems aren't simple. They certainly include rejecting the people who are outright being malicious. They also include a willingness to say, "Gosh, I keep using this didn't-mean-anything excuse. Maybe I need to start instead meaning to not do it."

oneplane6• 6 years ago

The issue with all of this, is that it's not just one company with a group of all the parts that design, create, test and sell this stuff.

Most of it was designed somewhere, built somewhere else, integrated yet somewhere else, and then simply bought out of a catalogue bulk, shipped to a warehouse and sold in smaller bulk to stores. The products and people that could have detected a problem are all over the world, and have seen every possible version of a human and still they end up in use with the problems they have.

If the pattern were constrained to fit a scenario as you describe then it would probably be something we can do something about. But just like in enterprise software contracts, the distance (both mentally, physically and legally) between all of this is not allowing for much change for any single actor. And I'm pretty sure the manufacturer of a sensor that doesn't even know where it ends up 10 layers deep in the supply chain is going to be remotely aware of this.

In practical terms we're seeing an anti-pattern and we try to personify it so we can express our dissatisfaction, but that's just as practical as shouting at the sun for not damaging all skin types at similar rates. There is no single source of the fault and there is no person or company or something like that, that is 'responsible' for it.

One possibility is to have some sort of consumer organisation test products on a diverse set of humans and report on products that lack support for certain shapes, colours or sizes, but other than that I have no idea how we'd adjust multiple people at multiple companies in multiple countries with multiple cultures.

klyrs2• 6 years ago

If you have stairs leading up to the porch of your house, they aren't there to prevent people with disabilities from entering. They're there because it's a cheap and convenient solution.

If the building is commercial, not residential, you'll find yourself on the business end of an ADA complaint. Not because of malice, but because of neglect.

newacct583• 6 years ago

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved.

The point isn't that people have different skin. It's not even that this one common device works better for some people than others. It's that the practice of medicine, having grown up around this cheap and effective device, turns out to be discriminatory in ways that we wouldn't naively expect.

That's worth talking about, because it's a problem that should be fixed.

apply Hanlon's razor when in doubt - it's not always malice like people often assume

Nowhere in the article is "malice" alleged. You added that part yourself, precisely because the bias described has infected you too. Reflexively arguing that "black people get bad O2 readings" is an argument that shouldn't be made in journalism isn't helpful to anyone, most especially the patients needing better care.

oneplane6• 6 years ago

My point wasn't to suggest that skin tones are the only thing in the world that aren't considered by all the people during all product design phases. It was more that we know that this is a problem in our bubble of communication, we know it just as much that we know that grass tends to be green (or yellow-ish when it doesn't rain for a while and the sun is out).

It's more that having a higher quantity of a known problem doesn't make the problem go away, and neither does trying to find a single source or single instigator of a systemic problem. It's not a new problem either.

If we take skin color and replace it with finger length for devices that require you to grip it, or with distance between the eyes for things that you look through, we have the same issue. In that regard this is just another facet of ergonomics in design. Even if we were to test for a lot of things in a lot of places in the supply chain you still can't catch everything.

jeffreyrogers3• 6 years ago

And people with sweaty hands. And people with low iron levels. There's this push to show that everything is biased against certain classes. I'm sure some of the examples are true, but I can't imagine that constantly seeking evidence of problematic issues is good for individuals or society.

krapp2• 6 years ago

Seeking evidence of problematic issues is how society identifies and eventually addresses such issues. Do you believe society would benefit from ignoring evidence or pretending those issues didn't exist?

jeffreyrogers3• 6 years ago

I agree that seeking evidence is how issues are addressed. The problem is that evidence is being sought in a particular way, which systematically biases which evidence is available for consideration.

Edit: there is a further problem that because many interpretations of the same facts are possible if you go looking for certain things you will very likely find them. While someone with different initial beliefs will find evidence of something else. All from the same facts.

rzmnzm• 6 years ago

A company I worked for decided one day to purchase budget fingerprint readers for door access. They were installed with much fanfare.

Unfortunately it turned out that the fingerprint readers were racist, and refused to allow black people into the building.

They were swiftly removed.

GenerocUsername• 6 years ago

measurement devices do not have 'bias' they have 'tolerance'. Give them some credit

GaryNumanVevo2• 6 years ago

Two separate things. The tolerance for blood oxygen is centered around sensor readings from white skin. Which is a bias.

readams• 6 years ago

It's maybe not too surprising that a device that works by shining a light through your skin is affected by pigments in the skin.

GaryNumanVevo2• 6 years ago

That's not supposed to be surprising, it's the fact most pulse OX meters are known to not work well for darker skin and it simply wasn't fixed.

wombatmobile• 6 years ago

GUIDELINES FOR SPO2 MEASUREMENT USING THE MAXIM® MAX32664 SENSOR HUB

The SpO2 measurement performance of a device must be verified before the device is released to the market. The U.S. Food and Drug Administration (FDA) suggests using standards presented in the following:

ISO 80601-2-61:2017 – Medical electrical equipment -- Part 2-61: Particular requirements for basic safety and essential performance of pulse oximeter equipment

Pulse Oximeters – Premarket Notification Submissions [510(k)s] Guidance for Industry and Food and Drug Administration Staff

According to these regulations, manufacturers need to declare the calibration range, reference, accuracy, methods of calibration and range of displayed saturation level. Furthermore, for the performance assessment, the FDA requires at least 200 data points equally spaced over a saturation range of 70% to 100%. Test subjects should have different ages, gender, and skin tones. For instance, the FDA requires that at least 30% of the volunteers must have dark skin pigmentation. The overall error or the root mean square error (RMSE) must be below 3.0% for transmissive pulse oximetry and below 3.5% for reflective pulse oximetry.

khuey• 6 years ago

Seems like it would be better if instead of merely an overall error of at most 3% they required an error of at most 3% in the subpopulations of interest too.

DoofusOfDeath2• 6 years ago

IIUC, dark-skinned people have higher covid19 mortality rates, and physicians usually(?) use this kind of oxymeter in their decision-making. I wonder if there's a connection.

sangnoir3• 6 years ago

The article also mentions that nonwhite people have a higher rate of their trouble with breathing being dismissed as due to anxiety compared to white people... It's possible there is a layer of biases, and at each step, there's a risk of not getting timely medical intervention resulting in poorer outcomes.

DoofusOfDeath2• 6 years ago

Right. I wasn't trying to imply that the oxymeter bias explained the entire difference in outcome rates. I'm just curious if it matters enough to warrant further study.

greenbush• 6 years ago

Here's a good article that discusses why Black Americans have higher mortality rates:

"systemic racism is as much a part of biology as genomes are: The conditions in which we develop—including limited access to healthy food, exposure to toxic pollutants, the threat of police violence or the injurious stress of racial discrimination—influence the likelihood that any one of us will suffer from high blood pressure, diabetes or serious complications from COVID-19"

https://blogs.scientificamerican.com/voices/racism-not-genet...

AlanYx• 6 years ago

This article doesn't seem particularly balanced. Unless I'm missing something, the author doesn't really give any generalized details about the amount of bias (for example, mean bias).

The author cites one study where doctors noticed "a bias of up to 8 percent", which sounds high, but if you click through to the link provided, it says:

The mean bias (Spo2 − Sao2) for the 70%–80% saturation range was 2.61% for the Masimo Radical with clip-on sensor, −1.58% for the Radical with disposable sensor, 2.59% for the Nellcor clip, 3.6% for the Nellcor disposable, −0.60% for the Nonin clip, and 2.43% for the Nonin disposable.

Basically, the amount of bias varies by manufacturer and type of device, and it is not always a positive bias as implied by the text of the link.

This seems to be more of a case of measurement error bounds inherent to any measurement device, rather than "a popular medical device encod[ing] racial bias" and all the other rhetoric in the article.

pj_mukh• 6 years ago

Interestingly, the new Apple watch will supposedly have a pulse oximeter of sorts [1]. I was looking forward to that. Now I wonder how it biases as well, especially given the wide distribution of apple watches and them being used as a defacto emergency health monitoring device [2]

[1] https://www.tomsguide.com/news/apple-watch-6-blood-oxygen-mo...

[2] https://9to5mac.com/2020/07/01/critical-heart-disease/

dafoex• 6 years ago

Sorry if I'm not demonstrating adequate sympathy, but all I can thing of from reading this headline is "oh dear, a person with a genetic adaptation that prevents some wavelengths of light from penetrating to deeper layers of the skin will find it hard to make some wavelengths of light penetrate to deeper layers of the skin"

As a nonmedical person, this seems blindingly obvious - to the point I could reference the "floor is made of floor" meme - so it would surprise me to learn that professionals haven't considered this downfall of the technology when taking their readings. Judging by the discussion here, I don't think I'm going to be surprised today.

0xdeadb00f• 6 years ago

I believe if we were given the race of the developers this bias would become clear - nothing against the developers or their race, I'm not implying the developers are racist in any way.

What I am saying is that systems developed by a team with a majority of single race is bound to have biases purely because of the lack in opinion diversity while testing the system.

But I could be completely wrong...

Edit: I pressed send by accident, I never meant to actually post this comment but I'm leaving it up anyways.

Pulse oximeters give biased results for people with darker skin

Related Stories