NTP Debugging Unreachable Time Sources

If ntpd has been configured to poll a specific time source, but the 'reach' value in the output of 'ntpq -p' is 0, then there may be different reasons.

First please note that a time source is considered reachable, if it is accessible (i.e. time and status can be read from it) and the status of the time source indicates that the source itself is synchronized.

Also keep in mind that an NTP daemon may not claim to be synchronized immediately after startup, but only after it has been running for some time and synchronized itself to some time sources that in turn claim to be synchronized.

See NTP Basics - Startup Behavior for details.

If the time source is a reference clock (e.g. a GPS/GNSS receiver), make sure

• the receiver is synchronized to its input signal

• the NTP daemon has been properly configured to read the time from that time source

• the NTP daemon has permission to access the reference clock, which can be denied e.g. by tools like SELinux or AppArmor on Linux

If the time source is an upstream NTP server:

• run 'ntpq -p' on the server, if possible, to see if the server is indeed synchronized

There are several tests that can be run to check why an upstream NTP server is not reachable. Please keep in mind that usually

• request packets are sent from the client to the server, which have to arrive at the server

• the server sends a reply packet back to the client whenever it has received a request packet, and the reply packet has to arrive at the client

First ping the NTP server to see if it is reachable at the network level:

ping host

Where host is the IP address or hostname of the NTP server.

If the ping fails, this can be due to a routing problem, where the request packet didn't arrive at the server, or the reply packet from the server didn't arrive at the client.

If a hostname instead of an IP address is used for the host in the command above, another problem can be that the DNS system of the network hasn't been configured properly to resolve the hostname to the correct IP address.

So if the command doesn't work using the hostname, retry if it works using the IP address to specify the remote machine.

If the ping test succeeded, the next question is whether outgoing or incoming packets are blocked by a firewall on the local machine, or on the remote machine, or somewhere on the network between them.

Usually, the firewall has to be opened for incoming and outgoing UDP packets on port 123.

The ntpdate tool that usually is included in the NTP software packages can be used to test this.

The command

ntpdate -d host

can be used to send a couple of NTP requests to a remote machine, and even to the local machine, and display the results.

If the connection succeeds, the output is something like:

Looking for host 172.16.100.252 and service ntp
host found : 172.16.100.252
transmit(172.16.100.252)
receive(172.16.100.252)
transmit(172.16.100.252)
receive(172.16.100.252)
transmit(172.16.100.252)
receive(172.16.100.252)
transmit(172.16.100.252)
receive(172.16.100.252)

server 172.16.100.252, port 123
stratum 1, precision -18, leap 00, trust 000
refid [GPS], root delay 0.000000, root dispersion 0.000183
reference time:      e1e7cce0.0960148d  Fri, Feb  7 2020 12:47:44.036
originate timestamp: e1e7cce4.09ebe83a  Fri, Feb  7 2020 12:47:48.038
transmit timestamp:  e1e7cce4.09ca1081  Fri, Feb  7 2020 12:47:48.038
filter delay:  0.02586    0.02583    0.02585    0.02576   
               —-       —-       —-       —-      
filter offset: -0.000046  -0.000028  -0.000046  -0.000014 
               —-       —-       —-       —-      
delay 0.02576, dispersion 0.00000, offset -0.000014

 7 Feb 12:47:48 ntpdate[23714]: adjust time server 172.16.100.252 offset -0.000014 sec

On success, the output shows both transmit and receive events, indicating that replies have been received, and the last line shows the computed offset between the time on the remote server and the own system time. This also indicates that the remote NTP server is synchronized and can be used.

If the command fails, the output looks similar to:

Looking for host 172.16.100.253 and service ntp
host found : 172.16.100.253
transmit(172.16.100.253)
transmit(172.16.100.253)
transmit(172.16.100.253)
transmit(172.16.100.253)
172.16.100.253: Server dropped: no data

 7 Feb 12:49:48 ntpdate[23875]: no server suitable for synchronization found

Here are only transmit events, but no receive events, so the summary says:
no server suitable for synchronization found,

So if the server is up and synchronized, and can be pinged, there must be a firewall which blocks the packets.

Please note the no server suitable message can also be displayed if replies from the server could be received, but the server is not synchronized, and thus won't be used by ntpd.

Please note also that this command can really apply an adjustment to the own system time.

Even if the test above succeeded, there can still be a firewall problem.

The ntpdate -d or ntpdate -q commands send requests to the NTP port 123 on the remote machine, but the requests are sent from a random, unprivileged port on the local machine.

However, ntpd also sends requests via the well-known NTP port 123, so if the firewall prohibits this, the 'ntpdate' command above may work, but ntpd may not.

So an additional test can to run ntpdate once more, but also sending its request via port 123. This is done if ntpdate is run without parameters -d, -q, or -u:

ntpdate host

Please keep in mind that it's not possible that 2 programs use the same network port at the same time, so ntpd or any other program that uses port 123 has to be stopped before this command can be used.

This command does not print any debug output like transmit or receive, but the summary is the same as in the previous test: The time difference is displayed if the program succeeds and replies from the server can be received, and the no server suitable message message is displayed if no replies are received, or the server is not synchronized.

If the previous test succeeded, but this test failed, there must be a firewall that blocks either the request packets or the reply packet.

If even sending and receiving packets on the privileged port basically works, but ntpd doesn't accept the replies from a server anyway, there can still be some restrictions in the configuration file /etc/ntp.conf which prevent the NTP daemon from accepting the packets from a specific server.

To check if this is the case, open the file /etc/ntp.conf in a text editor, comment out all restrict lines by putting a '#' character at the beginning of those lines, and restart ntpd.

If that works, re-enable the restrictions step by step by removing the '#' character from the beginning of a specific restrict line, and restart ntpd after each change to see if it still works, or not.

A summary and detailed description of the restrict keyword can be found on the NTP documentation page at

• https://www.meinbergglobal.com/download/ntp/docs/html/accopt.html#restrict

–

— Martin Burnicki martin.burnicki@meinberg.de, last updated 2023-01-25