NTP Server Reverts to 2000

From the ISC Diary:

A few people have written in within the past 18 hours about their NTP server/clients getting set to the year 2000. The cause of this behavior is that an NTP server at the US Naval Observatory (pretty much the authoritative time source in the US) was rebooted and somehow reverted to the year 2000. This, then, [propagated] out for a limited time and downstream time sources also got this value. It’s a transient problem and should already be rectified. Not much really to report except an error at the top of the food chain causing problems to the layers below. If you have a problem, just fix the year or resync your NTP server.

Doesn’t look like this impacted me at all, if it did logs and graphs would look funny. This however is quite freaky. Curious if this had any bigger impacts like financial transactions. You would think they would have some sort of check for strange NTP updates as a clock drift of 12 years is out of the ordinary, but anything is possible.

I’d also be curious to know how that server reverted to the year 2000. Perhaps it was something as simple as the CMOS battery died.

Leap Smear

On how Google deals with leap seconds:

The solution we came up with came to be known as the “leap smear.” We modified our internal NTP servers to gradually add a couple of milliseconds to every update, varying over a time window before the moment when the leap second actually happens. This meant that when it became time to add an extra second at midnight, our clocks had already taken this into account, by skewing the time over the course of the day. All of our servers were then able to continue as normal with the new year, blissfully unaware that a leap second had just occurred.

Good idea. The second itself is meaningless. Spreading it out is much better/easier than accommodating for it in the rest of your stack.