Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Bruised by past mistakes, tech firms brace for 'leap second'

Tim Hornyak | June 30, 2015
Just before the stroke of midnight Tuesday Coordinated Universal Time (UTC), computerized clocks around the world will pause for a moment to squeeze in an extra second.

Just before the stroke of midnight Tuesday Coordinated Universal Time (UTC), computerized clocks around the world will pause for a moment to squeeze in an extra second.

The leap second, as it's called, is needed to keep UTC in line with solar time. The two get out of whack due to changes in the earth's rotation, and 25 leap seconds have been added to clocks since 1971. Network Time Protocol (NTP) helps regulate the official time among Internet servers, keeping it in sync with UTC.

But the last leap second in 2012 took some IT companies and other firms by surprise, and caused websites including LinkedIn and Reddit, as well as Qantas' passenger reservation system, to crash. The problems involved unpatched Linux OS kernels, Hadoop instances, Cassandra and MySQL databases and Java-based programs.

Linux systems in particular were the focus of discussion after the 2012 leap second, as the bug caused everything from slowdowns in performance to overactive systems that led to CPU locks, Ron Pacheco, director of product management at Red Hat's Platforms Business Unit, said via email.

This time around, however, vendors say they are better prepared.

"Our small-scale tests are promising, and we'll be watching during the event to quickly respond to any unforeseen issues that may arise," a spokeswoman for Reddit said via email.

A spokeswoman for LinkedIn, meanwhile, said clock adjustments are being made to prevent any problems.

Qantas, one of the first major companies affected by the leap second in 2012, was hit by computer outages that delayed flights due to the effect of the Linux bug on the Amadeus reservation system, produced by Spain's Amadeus IT Group. The system is used by dozens of airlines around the world.

"We have sought and received assurances from Amadeus that they have taken action to make sure that the same problem does not happen again this year, and we're confident that it won't," a Qantas spokeswoman said via email. Amadeus did not respond to requests for more information.

There are several methods of dealing with the leap second. Google, for instance, implements a "smear window" centered on the leap second. To ensure that its NTP servers are in sync with the extra second, they are slowed, or "smeared," by about 14 parts per million.

"Twenty hours later, the entire leap second has been added and we are back in sync with non-smeared time," Google Engineers Noah Maxwell and Michael Rothwell wrote in a blog post last month.

Another approach is to have servers simply count the 60th second twice at 23:59:59 UTC, Red Hat's Pacheco said.

The leap second will kick in at 9 a.m. Wednesday in Japan, just as businesses start work. Dominant mobile carrier NTT DoCoMo has programmed its servers to slightly extend the length of each second over a couple of hours before 9 a.m. to stay in sync with UTC, a spokeswoman said. Rival SoftBank and major messaging app Line said they are also taking countermeasures.

 

1  2  Next Page 

Sign up for CIO Asia eNewsletters.