Multiple Boeing 787s in China experienced GPS 20 years rollover issue

https://twitter.com/ChinaAvReview/status/1114802018919411712

by jcfrei • 7 years ago

126 59 7 years ago

PaulHoule3• 7 years ago

Funny I got an old Tom Tom for $10 a few weeks ago, too old to get software updates, and I was pleased to see that it worked just fine after the rollover.

420codebro• 7 years ago

It isn't really the Nav/Positioning that has issue- but the time. Check the time on your Tom Tom unit. Is it 2019?

bookofjoe2• 7 years ago

Sort of related: my Sony TV, circa 2013, ALWAYS has this opening message (even if there's been no power outage that I've noticed): "Power outage. Time needs to be reset." I've been ignoring it since 2014 because it takes about 5 minutes and a half-dozen menus correctly sequenced (but only after multiple FAILs) to reset correctly.

labster2• 7 years ago

At the [dead] commenter: Timekeeping is an integral part of how the Global Positioning System works, to the extent that relativistic time dilation is a factor. You simply cannot use a GPS receiver with a broken time implementation, at least not without significant accuracy issues.

myself248• 7 years ago

But internal time is not the same as display time.

Internally, the week counter just rolled over. That's fine; all the almanac data is handled in that same format internally. Some receivers may need a reset when this counter rolls, and may need to relearn the almanac/ephemeris from the sky, but once that's done, Sunday of week 0 is just the same as any other Sunday of any other week 0.

Navigation works just fine because it's all internally consistent.

But display time, ahh, now that's where things get interesting. Sunday of week 0 could be displayed as 2019 April 7, or it could be displayed as 1999 August 22, or it could be displayed as 1980 January 6.

Knowing which epoch you're in is actually considerably trickier than working consistently internally, since there's no information in the satellite signal to tell you this, by design. It comes from applying an external offset. Receivers have lots of ways to do this, and quite a few simply have it baked in and cannot handle a rollover, so as far as they're concerned, week 0 is in 1999, period. They'll navigate just fine but they'll display the date wrong, because they're applying the wrong offset.

dsfyu404ed• 7 years ago

Any tips for someone looking for similarly priced antiquated GPS's to shove in the glove-box of all the cars in their fleet?

PaulHoule3• 7 years ago

There were four of them here:

https://ithacareuse.org/

there also was a $5 pair of Bose headphones that I was stupid to not buy the instant I saw them.

The Tom Tom I have is too old to get updated maps, but it's not a problem driving around in upstate NY. In New Hampshire it might be another thing, but where I live they build new buildings but not new roads.

gwbas1c• 7 years ago

Dumb question: What is the rollover issue?

usermac• 7 years ago

I had to look it up as well. I found this link: https://www.orolia.com/resources/blog/lisa-perdue/2018/gps-2... From the text comes: "However, the field that contains the week number is a 10-bit binary number. This limits the range of the week number to 0 – 1023, or 1024 total weeks."

Someone1234• 7 years ago

Not a dumb question at all.

GPS used ten bits for datetime. With a week counter and seconds into that week since 6th of January, 1980. After 1024 weeks it resets back to 0. This has happened once already on 21st of August, 1999. This is the second rollover/reset of the week counter on 6th of April, 2019.

Newer versions of GPS messages use thirteen digits instead of ten digits allowing for a larger week counter (granting 8,192 weeks), causing the first rollover not to occur until the year 2137.

https://en.wikipedia.org/wiki/Global_Positioning_System#Time...

kevhito• 7 years ago

Instead of adding 3 bits, maybe they should have dropped 3 bits. A rollover every ~20 years is just asking for latent buts to blow up. A rollover every ~2.5 years is just business as usual. Go big or go small. Don't design things to break just after you retire but before your coworkers do.

throw0101a4• 7 years ago

GPS used ten bits for datetime.

Nit: GPS uses 29 bits for "datetime".

10 bits for the week since the epoch, and 19 bits to count the number of 1.5s intervals that have passed since the start of the week. Yes, 1.5s intervals.

See Section 2.3.5:

* https://www.gps.gov/technical/ps/1995-SPS-signal-specificati...

js22• 7 years ago

https://news.ycombinator.com/item?id=19593227

petee2• 7 years ago

I don't get how some planes were affected, but not all; Every 787 in China should be running the same software version, let alone the world, no?

Are there any reasons a fleet plane might be using a different version? I'm coming up short on reasons, other than language, or maybe special hardware modifications?

j16sdiz• 7 years ago

They have regular software updates. Just like Windows update, nobody do that regularly

petee2• 7 years ago

Haha, that's surprising though, since these are important systems, if there is an update, doesn't it mean something really needs to fixed, i.e, gps rollover? Any bug left unpatched seems unacceptable

Maybe I have more faith in the 'system' than I should...

Yizahi• 7 years ago

I concur. For example in telecom getting operators to update anything is just a pain. Unless they are seeing service affecting issues, they are totally fine running decade old software. Also sometimes if they have lets say version 5 and current is version 12, latest, and they need some fix, then depending on the size and wealth of the company they may force manufacturer to create version 5 plus fix, making it a new branch just for them, even if all versions 6 to 12 were already tested in production.

netsharc• 7 years ago

iPhones on China have interesting "hacks" to comply with laws (GPS has some restrictions: http://nowiknow.com/the-problem-with-chinese-gps/), and they're not allowed to have the Taiwan flag emoji.

I also noticed the other day that Google Maps shows static images for S. Korea (not the fancy WebGL stuff). Also due to laws.

izacus2• 7 years ago

How does this user-facing map change translate to avionics though?

js22• 7 years ago

Related discussion which explains the rollover issue:

https://news.ycombinator.com/item?id=19593227

mschuetz• 7 years ago

What's the reasoning for the current GPS time format and it's resulting issues? Can't we just use a 64bit integer timestamp? That would give us roughly 580 years in nanosecond resolution?

crooked-v• 7 years ago

What's the reasoning for the current GPS time format and it's resulting issues?

Somebody wanted to save money when building the original satellites in the 1970s.

bmurray7jhu• 7 years ago

The GPS signal format was was designed in the 1970s. Any change to the signal format must be backwards compatible with legacy receivers.

There have been some efforts to modernize the GPS signal format. In 2014, select GPS satellites began broadcasting an additional message (CNAV) with a 13 bit week number, allowing a ~157 epoch.

jhayward• 7 years ago

What's the reasoning for the current GPS time format and it's resulting issues?

GPS satellites and receivers last on the order of 20 years so it takes a long time to age out of legacy support. The latest generation of satellites, GPS-III, have a new week number format that allows for 13-bit week numbers (120 year rollover).

The reason for the low number bits in week number is simple, it's broadcast on a very low bit rate (50 bits per second) and every frame has the week number in it. So you get faster fixes if you don't pad out with useless zeros.

throw0101a4• 7 years ago

The reason for the low number bits in week number is simple, it's broadcast on a very low bit rate (50 bits per second) and every frame has the week number in it. So you get faster fixes if you don't pad out with useless zeros.

It should be noted that the coders could have used data windowing and pivot dates to make certain assumptions:

When ntpd(8) receives an unresolved timestamp from an upstream server, that timestamp could be based in any era prior to the current wall-clock date - but by definition we may not know the wall-clock date with confidence yet (not having achieved sync).
To resolve this ambiguity, NTP also uses an internal pivot date. It chooses the era that would minimize the absolute value of the time difference between the resolved timestamp and its internal pivot date.
This resolution technique is part of the NTP specification. It will normally resolve to the current era, but may in cases where the pivot is within a half-cycle of a rollover time resolve to either the previous or the next era.
An ntpd(8) instance’s pivot date will be the date it was compiled and built.
It is fairly easy to see that this guarantees correct resolution if the pivot dates of the communicating ntpds are within a half-cycle (88 years) of each other, even if there is a rollover between the pivot dates.

* https://docs.ntpsec.org/latest/rollover.html#ntp_pivots

* https://en.wikipedia.org/wiki/Date_windowing

michaelt• 7 years ago

GPS was designed a long time ago, and the data rate for ephemeris and almanac data is.... 50 bits per second.

That's not an exaggeration for comic effect. There's literally 50 bits per second from each satellite. 6.25 bytes.

So the navigation messages are packed extremely densely. To the point that plenty of numbers don't even start on byte boundaries.

leecb• 7 years ago

The age isn't the only explanation for the low data rate- it's not simply low tech.

GPS relies on receiving extremely weak signals, often with a very limited receive antenna. This means there's a real trade off- you can send a limited amount of data over a given communication channel. Increasing the data rate might require increasing the transmit power / antenna gain of the GPS satellites, or requiring bigger / better receive antennas on GPS receivers.

Here are the parameters involved: https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theore...

userbinator• 7 years ago

The 787 didn't even exist when the GPS time last rolled over, so it's quite unusual to see it have this problem --- unless Boeing reused the software from a previous version, which is not surprising given the regulations of the industry, but that would imply other older models also have the problem.

acqq• 7 years ago

The 787 didn't even exist when the GPS time last rolled over, so it's quite unusual to see it have this problem

It's not as unusual as you'd think:

https://blog.fosketts.net/2019/04/06/gps-time-rollover-failu...

There you see a nice summary of the problem and an example a Mercedes car from, if I understood, 2008 experiencing the same type of problems. Also discussed on HN:

https://news.ycombinator.com/item?id=19593227

It's like having a Y2K bug software and saying "but that software didn't exist before 1900." It doesn't matter when the "epochs" are, it's just about what the design covered or didn't.

I've personally fixed the trading software which failed to correctly implement the correct rollover of the decades (i.e. the transition from year 9 to year 0 in the names), not to mention Y2K. It was simply tested before only "in the same decade."

It's definitely a failure in the development process, for the software that is supposed to work in some bigger time frame, and not only shortly.

syockit• 7 years ago

I'm rather interested in what kind of software developed after Y2K still exhibiting Y2K bug, and how it affects the usage. Is it made with the assumption that it's going to record only post-Y2K dates, and somehow user manages to input pre-Y2K date?

theclaw• 7 years ago

It's definitely a failure in the development process, for the software that is supposed to work in some bigger time frame, and not only shortly.

This is why I've recently started abstracting time/date calls in all my work so that I can test using arbitrary dates/times rather than using the various "now" library calls directly.

This has already had the side-benefit of allowing me to deal with stuff like the time arbitrarily going backwards when NTP synchronises the clock. I don't typically rely on wallclock time for sequencing, this just keeps things a little more robust.

Thermolabile• 7 years ago

Can someone give me a source of it?

AnssiH• 7 years ago

Someone on reddit said yesterday a KLM flight (apparently B777) was grounded because of incorrect date (in some Honeywell equipment?): https://www.reddit.com/r/sysadmin/comments/babyvv/anyone_hit...

Aromasin2• 7 years ago

We had some Honeywell engineers at my workplace for a system integration meeting. Over lunch I got the opportunity to talk with them. This came up in conversation. Apparently it's very much all-hands-on-deck at the moment - almost every system they've currently got on-board Boeing aircraft is being assessed, and they're in all hours of the day to get new revisions through. I do not envy them right now.

mehrdadn• 7 years ago

Is it because they didn't see the GPS rollover coming up (it was in the news though)? Or did they not think their products would be affected?

Multiple Boeing 787s in China experienced GPS 20 years rollover issue

Related Stories