Leap seconds: Causing bugs even when they don’t happen

https://berthub.eu/articles/posts/leapseconds-expose-bugs-even-when-they-dont-happen/

by ahubert • 5 years ago

228 172 5 years ago

berthub.eu

FabHK2• 5 years ago

What we'd like to have:

1. Use SI seconds

2. Keep in sync with earth rotation

3. Avoid leap seconds

Pick any two:

- UT1: 2, 3 (not 1) [https://en.wikipedia.org/wiki/Universal_Time]

- TAI: 1, 3 (not 2) [https://en.wikipedia.org/wiki/International_Atomic_Time]

- UTC: 1, 2 (not 3) [https://en.wikipedia.org/wiki/Coordinated_Universal_Time]

ahubert• 5 years ago

Hence, the giant leap second gyroscopes! :-)

contravariant5• 5 years ago

This is neither here nor there, but "leap second gyroscopes" has turned up some of the weirdest google search results I've seen. A few examples:

Downward movement of work quality? He drew in step a leap second? Gyroscope and accelerometer. Gunfire this weekend outdoors and enjoy shopping here!
Successful troll was a leap second? Gyroscope and accelerometer. Blissful for you. Both inconsequential from a class. Miracle whip classic macaroni salad?
Had lent thee all a leap second? Gyroscope and accelerometer. Shah is threatening hearing or sound. Been fired from their roost and lay siege to the renovation ...

Now it's not unusual for webpages to pad their text in order to trick search engines, but these are downright weird.

zokier3• 5 years ago

Afaik additional problem with UT1 is that it's near impossible to tell UT1 time in real-time, because you need to do astronomical observations. So it's kinda impractical as civil time.

fanf23• 5 years ago

The rotation of the earth is predictable enough in the short term that it is forecast a year into the future: see IERS Bulletin A https://www.iers.org/IERS/EN/Publications/Bulletins/bulletin...

thaumasiotes3• 5 years ago

As far as I remember from earlier discussion, the reason that problems arise is that the posix standard defines a "day" as 86400 seconds, so leap seconds need to be erased from history after they've happened. This doesn't make a lot of sense; on March 1st, we don't start pretending that February 29th never happened.

zokier3• 5 years ago

Well, POSIX time is an additional layer of stupidity, but it certainly not the only source of problems; notably these GPS/GNSS related problems do not have anything to do with POSIX time

phicoh2• 5 years ago

The problem is that posix needs the conversion between 'posix time' (seconds since 1970) and the split out year/month/day/hour/minute/second format to work for the future as well.

Given that we don't know when there will be leap seconds in the future, this conversion is imposible if we want to take leap seconds into account.

So the solution is to either ignore leap seconds (as posix currently does) or have a regular rate of leap seconds (just like leap years). Unfortunately, the rotation of the earth is are not regular enough for a fix rate.

At the scale of a human life, leap seconds are completely irrelevant if you want to know the position of the sun.

So it is bizar that our civil time has leap seconds. Looking back, that seems to be a historical mistake.

dhosek• 5 years ago

on March 1st, we don't start pretending that February 29th never happened.

Speak for yourself.

qalmakka3• 5 years ago

Just stick with TAI and convert times to UTC when displaying them is required. The real crap here is the C and POSIX time_t type and its related functions, which are massively out of date (like the 70% of the C standard library). The ISO C standard committee is way too scared of breaking stuff, and when they add something is often utter garbage (look at the whole _s functions fiasco in C11).

TazeTSchnitzel2• 5 years ago

They should be scared of breaking stuff! If newer standards break things, those newer standards are unlikely to get wide adoption, which prevents actual progress.

In any case, while the POSIX and C standards need to accommodate atomic time, that's not enough. The software on top needs to switch to atomic timestamps too.

globular-toast4• 5 years ago

The problem with rendering TAI as UTC is you can't do it for future dates because you don't know how many leap seconds will have been added by then. For future dates you can either:

- store TAI: you'll always know how many SI seconds in the future it is, but you won't be able to accurately render UTC,

- store UTC timestamp: you'll always know what the time on the clock will be when the event happens, but you won't be able to accurately calculate how many seconds in the future it is.

The alternative is to ditch UTC. Make all wall clocks tick in SI seconds and render TAI dates in local time. This will always be correct. The only downside is you won't be able to accurately predict what position the Sun is in the sky when the event happens. But is this actually important?

layer83• 5 years ago

Just stick with TAI and convert times to UTC when displaying them.

That may not be suitable when you want to represent an exact point in time years into the future. You’re usually not interested in the exact number of elapsed seconds (TAI) in that case, but want to represent e.g. 2031-08-03T:00:00:00Z exactly.

The solution is to use different data types for elapsed time and for calendrical/time-of-day time. Software developers need to learn that those are not equivalent, and libraries, databases etc. need to provide appropriate data types that make the distinction clear.

vitus3• 5 years ago

Smear it! You sacrifice 1 on days with leap seconds (by 0.001%), but otherwise have all three.

https://developers.google.com/time/smear

michaelt• 5 years ago

Ah yes, for people who want to support leap seconds, but who can happily ignore a clock error of 0.5 seconds :)

contravariant5• 5 years ago

Kind of, but differences in time will forever deviate from the SI second so I don't think that really counts.

Also 0.0001% of the distance to a GPS satellite is on the order of 200m. Not impossible to work with, but not great either. Though I have no idea what the exact ramifications would be.

dooglius• 5 years ago

Who is the "we" who wants to keep "in sync" with the Earth's rotation to such high precision? Obviously having 3:30 pm suddenly become the middle of the night would be bad, but we're talking a delta of 18 seconds spread over almost 50 years... hardly big enough to be called "out of sync" in any noticeable sense. Maybe in a millennium or two we'll lose the cultural context of what "it's 5:00 somewhere" meant, but I'm sure the historians could add a blurb.

globular-toast4• 5 years ago

Agreed, 2 seems way lower priority than the other two. You already put your clock "out of sync" with the Earth's rotation every time you move East or West. Nobody actually cares about this. 1 hour timezones are generally granular enough for our needs.

throw0101a3• 5 years ago

Obviously having 3:30 pm suddenly become the middle of the night would be bad, but we're talking a delta of 18 seconds spread over almost 50 years... hardly big enough to be called "out of sync" in any noticeable sense.

And the Julian calendar was 'good enough' for a long time… until it wasn't and Pope Gregory XIII had to announce the skipping over of 10 days so that (e.g.) the Spring Equinox actually lined up with the season of spring more accurately.

I understand the sentiment of wanting to simplify things, but kicking the can down the road so it's someone else's problem is wrong in my eyes.

teekert3• 5 years ago

Why is it obvious that 3:30 PM bad as the middle of the night is bad?

I'm all for a universal world-wide time. then we'd just have to learn what time is middle of the night for what location, but we have digital aids for that. It solves all timezone bs.

My guess is is that we whine for about a month and then it becomes normal for ie. 14:00 to be middle of the night depending on you location.

quirino• 5 years ago

We don't even have to go through that situation. As soon as the delta reaches 30 minutes you just shift every timezone by one hour.

cowboysauce• 5 years ago

Who is the "we" who wants to keep "in sync" with the Earth's rotation to such high precision?

The military is a big one. ICBMs still use celestial navigation to determine their position and adjust their trajectory in flight.

wpietri4• 5 years ago

I don't specifically care about earth's rotation, but as a diurnal creature whose internal body clock synchronizes to the sun, I definitely care about the rhythms of natural light. The whole reason clocks were invented was to help people more precisely measure the day-night cycle.

So the question really becomes, "Who is the 'we' who wants to have a more theoretically pure timekeeping system that gets worse and worse at its primary purpose?" Turns out that's a pretty small audience.

sjburt3• 5 years ago

Astronomers and navigators. These also were the first people that needed accurate time and first people capable of measuring time accurately, so much of the expertise is in astronomical or naval institutions. This was still very relevant in the 1950s when the modern time systems were being developed.

There is a growing movement in the timekeeping community that agrees that keeping UTC close to UT1 is no longer important and that leap seconds should be abandoned.

account423• 5 years ago

TAI sounds perfectly reasonable for most systems and should be the default IMO - only when displayed to humans do we really need to care about syncing up with the earth's rotation.

lifthrasiir• 5 years ago

UTC with only leap hours will satisfy all three, assuming 2 can be slightly weakened.

zokier3• 5 years ago

You can even avoid leap hours if you are willing to have Greenwich not be at +00:00 timezone

contravariant5• 5 years ago

Clearly the most realistic solution is to slow down the rotation of the Earth when necessary.

richardwhiuk• 5 years ago

I think you mean speed up.

denton-scratch3• 5 years ago

One of the things we want is a timescale that allows us to put a timestamp on a contemporary event, with the hope that in 2,000 (or 200,000) years, they will be able to work out exactly how long ago it was we were referring to.

GMT failed disastrously at that; there have been about 12 mutually incompatible definitions of GMT over the years.

So, UTC should retain leap-seconds; that's what UTC means. A new timescale, without leap-seconds, MUST have a new name.

I'm betting the ITU will just change the definition, without changing the name.

themulticaster• 5 years ago

I think you're looking for TAI. UTC = TAI + Leap Seconds (unless I'm missing some minor detail).

You don't know how many seconds lie between arbitrary UTC timestamps in the future, but you do know how many seconds lie between arbitrary TAI timestamps in the future.

Uberphallus• 5 years ago

Worth mentioning, there's an ongoing debate about removing leap seconds, as they seem to cause more trouble than benefit they bring (UT1 and UTC being in sync).

Personally I've dealt with some unit tests around simultaneous use of multiple timezones (we had to show the timestamps in the local timezone of the respective event). One day a good chunk of them stopped working for no apparent reason.

After a lot of head scratching, I figured out an update of tzdata was responsible for it, since it added new planned leap seconds. Our underlying library was properly taking the leap second into account, but the test conditions weren't.

nly5• 5 years ago

At my work we have a system where dates are in a local timezone, where the timezone depends on the data ingress point, but times are in New York local time.

So many bugs occur when juggling 2 or 3 timezones simultaneously.

quietbritishjim2• 5 years ago

I won't name it, but I can guess where you work, as the place I once worked had the same problem and surely not many places have made the same mistake.

Worst is where the NY local time jumps back due to DST and you have two actual points in time represented by the same numerical NY local time. That manifests itself as an hour long gap in the data for the Australian stock exchanges!

eerikkivistik• 5 years ago

This could lead to an interesting situation, where in June of 2029 a multitude of double-spending attacks are initiated on various banking/financial software across the world, with the attack lasting a few seconds in total. What a headline that would be... Of course this depends on specific implementation, but I can see how this could happen to a wide array of implementations.

makeitdouble2• 5 years ago

Looking at how banks handled these kind of issues in the past, they'll shut down all operations for these few seconds and throw away anything that still came in these seconds as invalid.

It sucks for their customers and partners, but that would be a decent conservative option I think.

MiddleEndian• 5 years ago

Somebody should have told the financial institutions in Batman: The Dark Knight Rises that throwing away very obvious unwanted transactions was an option!

debarshri2• 5 years ago

One of the interesting corner cases of leap second is in traffic enforcement. It you are building section control traffic enforcement system, you measure the t1 at point on entry and t2 at exist. Because of you know the length of a section , distance over time difference becomes your speed. But when you have leap second, you might just get a fine for driving at jet-fuelled speed on your grandpa's car.

adrianN• 5 years ago

I really hope that traffic enforcement systems don't use timers that know the date.

debarshri2• 5 years ago

It is not just for timer, as part of evidence you also have to capture when that event occurred. There it does capture the date.

Faaak• 5 years ago

Supposing the speed-controlled road spans 1km, then if you go at 100km/hour, it will take you 36s:

  You have: 1km/(100km/hour)
  You want: s
   * 36

With a second less, your speed would be miscalculated to:

  You have: 1km/(36s - 1s)
  You want: km/hour
   * 102.85714

So not really something to worry about

seanc• 5 years ago

Unless the road span in question is a 10m intersection. Now if you went through at 30km/hour it took you 1.2s, but lose that second and now you went through at 180 km/hr.

Go through at 60km/hour and you left the intersection before you entered it. Does that mean the government owes you the fine?

foxhill2• 5 years ago

in the UK, speed traps (that aren't "average speed check" traps) operate over a distance of ~10-30 meters, depending on the road's speed. that second is very important.

edward• 5 years ago

Previous discussion: https://news.ycombinator.com/item?id=27944776

kortex• 5 years ago

(One of) the biggest pains with leap seconds is they are so infrequent and so I/O-bound, that they are almost impossible to test. What is less functional than relying on a random change in a radio signal below the noise floor coming through dedicated hardware, which is finicky and doesn't work well indoors (E-coated glass windows? Fuggetaboutit).

I think one thing that could have be easily done to help is emit leap-sub-seconds (100-500ms) more frequently, so we would have more chances to detect bugs per year. Alas, leap seconds are baked in as int8 so that's never gonna happen. That's also totally not the mindset when these systems were designed, which is kind of my point. The right kind of forethought could have avoid much of this pain.

Option 1: We have 2 time standards out of sync. Let's make an announcement yearly-ish and add 1 second of offset that everything has to obey.

Option 2: We have 2 time standards out of sync. Let's continuously track the offset to (whatever precision) and this offset is just always part of the correction (like we do time zones), rather than this value which you just assume is static 99% of the time.

trzy• 5 years ago

We had a leap second bug on the options trading desk I worked on that brought down our system. As market makers, we had an obligation to keep providing liquidity so this was a serious issue and exposed us to fines.

Our exchange co-located data centers used GPS for precisely synchronized timestamp generation but the firmware on some of the GPS hardware had a bug and failed to take into account a leap second. When that leap second was to be inserted, they became out of sync by a second.

We generated spot price feeds from each location and a component that consumed these feeds would check to ensure that they were not stale (any data more than 0.5 sec old could not be used as an input for pricing and trading). Well, a lot of exchange feeds started looking stale, and our system stopped quoting on said exchanges.

First there were some murmurs from the traders and within minutes the entire room hit a crescendo of panic. It took, I think, the better part of an hour to debug.

8ytecoder• 5 years ago

How did you resolve it?

omegalulw2• 5 years ago

They just needed to add leap seconds to the hardware timestamp after they debugged it.

hacker_newz• 5 years ago

Can you provide the model of the GPS hardware? I work in the same industry and have set up multiple NTP / PTP timer servers that have never failed to account for leap seconds. It's almost always the downstream software that crashes. See the 2012 leap second insertion as an example.

fhars• 5 years ago

I am still in favor of replacing the annoying one hour jumps when switching to and from DST by having a leap second nevery hour for 150 days. That would also force people to fix their leap second bugs.

platelminto• 5 years ago

Is there somewhere to read more about this? It sounds very interesting, are there any places/systems that are considering using this, or even talking about it?

_moof• 5 years ago

Offered for perspective/interestingness, not as an argument:

For anyone wondering why it matters so much that time be precisely linked to the rotation of Earth, I'll note that time is a fundamental component of navigation. When you do celestial nav you make corrections down to the second, including accounting for the number of seconds your watch gains or loses. So it's not just about putting timestamps in a database. There's a straight line (a rhumb line? har har) from "what time is it" to "where are we" and in that context losing a second means you get a different longitude. Because time in this setting isn't really time--it's an indication of how far east or west of the prime meridian the sun is.

bialpio• 5 years ago

Thanks, this helped me adjust my mental model!

omegalulw2• 5 years ago

Is there any practical navigation use case that needs enough precision to not be off 27 seconds from Earth's rotation and unable to use GPS?

drallison• 5 years ago

Leap seconds are a bad idea and should be abandoned. Computing the time between events by subtraction should work. And while we are at it, dropping time zones and daylight saving time would also make sense.

jameshart5• 5 years ago

So many comments on here asserting that leap seconds are bad and should be abandoned and that it would be simpler if they didn’t exist, but…

You realize this is about the GPS system, right? The thing about GPS satellites is that they are in space, orbiting around the earth. If the earth’s rotation speed changes their orbital speed doesn’t (not immediately, anyway).

GPS absolutely needs to adjust for the variable rotational speed of the earth - otherwise the GPS coordinate grid would gradually move relative to the surface of the earth.

So GPS doesn’t exactly need leap seconds but it really does care about how long a rotation of the earth takes which… amounts to the same thing.

daddylonglegs• 5 years ago

So GPS doesn’t exactly need leap seconds but it really does care about how long a rotation of the earth takes which… amounts to the same thing.

I don't think this is right. GPS explicitly uses a timebase that does not include leap seconds [1].

On the subject of the article: my modest and very reasonable proposal is that we apply a leap second every six months without fail, dithering between positive and negative leap seconds so as to remain close to sidereal time. That way we would flush out bugs every six months and wouldn't have them accumulate and hit us all at once.

Or we could be boring and use TAI or GPS time as the system clock every where and apply leap second corrections when we go from the system clock (currently UTC) to local time.

[1] https://en.wikipedia.org/wiki/Global_Positioning_System#Time...

jameshart5• 5 years ago

I mean, the article is literally about GPS satellites transmitting leap second data as part of their messages. So yes, the basic clock is just counting seconds but leap seconds are a pretty important component of the GOPS model.

a13692099933• 5 years ago

GPS absolutely needs to adjust for the variable rotational speed of the earth

Sure, but that's a spacial thing, not something that's relevant to timekeeping. (There's technically a change in relativistic time dilation, but it's less than a rounding error - about five millimeters per second at the equator makes 1.000000000002388402 versus 1.000000000002388457 if I've got my math right. (The relativistic time dilation from the earth's rotation is itself a rounding error.))

jameshart5• 5 years ago

And when you’re using GPS to find your location on the surface of the earth… spatial things matter, right? GPS is for figuring out your location, not what time it is.

You can use it to figure out what time it is, but that’s a side effect not the goal.

fanf23• 5 years ago

So, GPS involves a few things: precise clocks that can be used to triangulate position; a model of where the satellites containing the clocks are in space; and a model of where the earth is in space.

UTC has basically the same inputs (the clocks are on the surface instead of in orbit, but one of the main time labs [NIST] is in Colorado and its altitude needs to be taken into account), but the calculations and logistics, the timebase of the model, are all very different.

So you can’t derive UTC from the GPS ephemeris, because there’s a human in the loop who decides when leap seconds happen. So GPS needs to include UTC as a separate signal, alongside the ephemeris and everything else.

(In fact, if I remember correctly, GPS includes nanosecond-level corrections between GPS time and UTC, because atomic clocks are not completely perfect.)

nullc2• 5 years ago

Sorry, but your comment is just nonsense. :( GPS time doesn't include leapseconds: It's effectively TAI with a different offset.

GPS indeed has to correct for all sorts of orbital stuff, but that is done with equations on top of a stable atomic timebase. (And it has to be that way-- each satellite's orbital parameters are different).

jameshart5• 5 years ago

Knowing your location relative to the satellites is only so useful if you’re actually standing on the surface of the earth (or trying to fly to a particular point on its surface). You also need to know which way the earth is facing right now. And to do that you need to know how many times it’s gone round since some reference time - which varies, but amounts to ‘number of days taking into account smeared leap seconds’

beervirus• 5 years ago

Every time I read an article about time handling in software, it makes me glad that’s not how I earn my paycheck.

Leap seconds: Causing bugs even when they don’t happen

Related Stories