Medical devices: A ticking time-bomb

http://www.economist.com/blogs/babbage/2012/05/medical-devices

by bootload • 14 years ago

68 32 14 years ago

m104• 14 years ago

NTP is great and all, but...

When you're timing something that needs accuracy, you use a timer not a clock. If you need to use a clock, you use one clock not all the clocks. Yes, I work in the medical industry. Yes, we generally use timers not clocks. Yes, our clocks are all wrong too.

If you really do need a solution to the "time" problem (accurate time all the time), you need to issue everyone a smartphone.

Confusion• 14 years ago

Except that my iphone is usually off by a few minutes. I don't think Apple got their NTP implementation or usage quite right.

keeperofdakeys• 14 years ago

Most mobile phones usually get their time from the phone towers. Of course, this relies on them having the right time.

Interestingly the most effective way to get syncronised clocks is GPS.

tomwalker• 14 years ago

im a doctor also and the need for highly accurate timing is exceedingly rare.

A lot of medical timing is guess work - why are antibiotics commonly prescribed for 5, 7, 10 or 14 days? Whynot 6?

Automatic filing of EMR information would be important and the legal aspect would unfortunately demanc the greatest amount of accuracy.

Surely connected devices could use one centrally calculated timesource?

stewartbutler3• 14 years ago

> "Surely connected devices could use one centrally calculated timesource?"

... like NTP?

Sorry, but I'm not sure what point you are trying to make here. Anything with a computer, even a microcontroller, and basic network connectivity can use NTP, it is an extremely simple protocol.

Anything that doesn't connect to a network could be kept in sync via a wireless pulse/tick, though then you have security issues unless you cryptographically sign the clock signal. Though I'm not sure why anybody would try and screw up a clock signal like that other than general asshattedness.

That aside, I'm unsure why they are using local time for logging data anyhow. If the folks managing the hospital were rational, shouldn't there be a regulation to use GMT and 24-hour clock? My home town is right on the Eastern/Central timezone split, and I know it causes all kinds of problems in day-to-day life. I just hate that kind of ambiguity.

devicenull• 14 years ago

You technically don't even need accurate time, you need consistent time. If all your clocks are consistently slow by 10 minutes, there's less of an issue then if some of the clocks are slow by 10 minutes.

pak• 14 years ago

"the average error was a staggering 24 minutes"-- The average is a terrible measure of centrality for data that is unbounded at one end, because one outlier (the 42 year off clock) skews the measure disproportionately; the median would be much more appropriate to report. That's unusually sensationalist number crunching for The Economist.

I'll bet that out of the clocks that doctors actually use to decide on treatment (tallying every device is a bit unfair; if a blood pressure monitor on some cart in a closet has the wrong time of day who cares) 80% of them are within a few minutes of the right time. And at the end of the day, doctors are supposed to be able to reconcile conflicting information, it's part of their training. This is a little bit like tallying up every textbook lying around a high school and getting concerned that the average book is three editions out of date and has 142.8 errors--calm down, most of those are not in active use and have some faith in teachers to check their material.

kijin3• 14 years ago

The research tested 1700 devices. A single device that is off by 42 years (reset to Unix epoch?) would have skewed the average by over 9 days. So unless a number of other devices had similarly extreme errors in the opposite direction (and even if they did, I'm pretty sure the researcher would have only looked at the absolute magnitude of the error, not their direction), I think it would be safe to assume that extreme cases were excluded from the average.

Bnd you're right, 80% of them were within 30 minutes of the right time.

> "One in five were off by more than 30 minutes"

In fact, given that the article mentions daylight saving time as a major problem, I'm not surprised that the average error approaches half an hour. At any given time, a lot of machines would be running in the wrong timezone.

abecedarius• 14 years ago

Good point about the stats.

doctors are supposed to be able to reconcile conflicting information

They're supposed to wash their hands, too. (http://boingboing.net/2010/03/09/study-says-us-doctor.html) As a guy who makes dozens of mistakes a day, the ones I know about because a computer catches them, I'm really not reassured by the idea that human judgement makes up for machine error.

chrisbroadfoot• 14 years ago

Average is a fine measure. You've inferred that they used a mean average. The article doesn't say what kind of average they took.

Let's assume they did take a mean. If the mean error for 1700 devices was 24 minutes, the total error is 40,800 minutes. That's 28 days.

Therefore, they didn't include the outlier of 42 years in their average. It's most likely a median (or mode) average.

blahedo• 14 years ago

It's pretty unlikely to be a median; if 20% of machines are 30 minutes off and 24 minutes is a median, then 30% of machines are between 24-30 minutes off, which would seem to be a strangely narrow band.

More likely, the 24 minutes is a mean average after excluding outliers. One of the standard ways to compute outliers is to measure the inter-quartile range---the "distance" between the 25th percentile value and the 75th percentile value---and call everything that is more than 1.5*IQR past the 75th percentile an outlier. You can do this without computing a mean first, and you can use it to exclude values before computing a modified mean.

Dylan168072• 14 years ago

Time errors between ten minutes and an hour are probably the most dangerous, so most devices being less than half an hour off is barely a comfort.

nitrogen2• 14 years ago

I have two questions:

  Since the EMR is updated constantly, data from devices
  whose clocks are way off would simply never be recorded.
  Others might bury current information in older files.
  Worse, they may insert the data into the EMR when the
  patient they concern has left the given clinical
  environment and another has come in.

1. Why does the central EMR trust timestamps from the devices?

  Daylight saving time corrections twice a year require
  tedious manual tweaks that the MD PnP Program, an
  initiative supporting networking standards for medical
  devices, estimates cost American hospitals over $17m
  annually. At Massachusetts General, a patient-monitoring
  system deletes an hours' worth of data when rolling back
  from 2am to 1am every autumn, while drug pumps are kept
  permanently on standard time, so they are (at least) one
  hour off for half the year.

2. Why are they using time zones at all?

bmj• 14 years ago

As someone who works with devices which require good timekeeping, I'll add another question: if the devices are communicating with a central server, why doesn't that protocol include a time update from the server?

famousactress4• 14 years ago

Hi. Written EMRs and built medical devices (10-12 years exp between them). In many cases messages from devices and external systems are processed asynchronously, so there isn't much choice but to trust remote timestamps.. I may be getting data minutes or hours later. Just a distributed system challenge.

The time zone issue is a human one. Historically clinicians schedule doses at "Every 3 hours" or "Once per day, 8am". Humans figure out what that means, but a dose scheduling system that shifts 8am an hour after daylight savings (because it interpreted the schedule as "every 24 hours" is going to run aground with nurses when it tells them they're late). However, every 3 hours is pretty obviously an interval and likely shouldn't shift by 30% of the interval!

This isn't a distinction that clinicians are used to making, but as someone who had to write some of the first dose-scheduling systems focused on clinical accuracy, we ran into this challenge and did a lot of thinking about which schedules really meant 'intervals' of specific units of time and which ones meant 'due at these calendar times every day'.

It's an interesting example of a problem that introducing technology introduces/exposes while showing up to fix other things.

nitrogen2• 14 years ago

So I'm guessing you might solve that by asking the device what its internal clock is at the time of synchronization, and adjust the logs by the offset between device time and EMR time (with manual device time changes included in the log)?

subwindow• 14 years ago

I wrote software for medical devices for 3 years. Saying 'just use NTP' is all well and good, except that getting Internet access inside of a hospital is usually impossible. So of course we offered a configurable setting so that they could use an internal or firewall-allowed server. I think one hospital took advantage of that setting. Everybody else was just wrong.

zdw• 14 years ago

NTP settings can be distributed by DHCP, which in a managed networked environment should be reasonably trustable.

Unfortunately Windows ignores this, and also sets the system clock to a timezone-specific time, not UTC as nearly all other OS's do.

stewartbutler3• 14 years ago

Well, if it could be regulated, you could require that the hospital put the NTP server or equivalent on IP over power line.

famousactress4• 14 years ago

Not sure how old your experience is, but there's increasingly less friction here.

NelsonMinar• 14 years ago

Configuring every single device to use NTP is a hassle. Do medical devices on an intranet use DHCP? DHCP is capable of handing out NTP server addresses, although I don't think it's very common. Another hacky option would be to override "pool.ntp.org" on the intranet's DNS server so it points to an internal server.

imcqueen• 14 years ago

why is the EMR automated at all if it isn't accurate?

famousactress4• 14 years ago

accuracy isn't boolean.

tokenadult• 14 years ago

Previous submission, with a few comments:

http://news.ycombinator.com/item?id=4013021

I'm glad that there is some more discussion of this issue, as medical devices are some of the main high technology products in my part of the United States, and my son (also a Hacker News participant) worked as an intern at one of the medical device companies here last year. Medical device businesses are much more regulated as an industry than are most Web-based SaaS businesses.

mmagin• 14 years ago

I just hope this doesn't result in poorer security between outside networks (including the internet) and medical devices. I'd argue there should be an "air gap" between the two, but I'm probably more paranoid than most.

Certainly it's fine to use NTP, but I'd want to use some kind of GPS-receiver NTP appliances at each site (possibly redundantly).

Medical devices: A ticking time-bomb

Related Stories