Questions about KVM-Clock and NTP for Ubuntu Guests

Wed Apr 24 21:18:57 UTC 2013

On Wed, 2013-04-24 at 11:16 -0500, Chris J Arges wrote:
> Hi everybody, I have a fairly in-depth set of questions about
> kvm-clock and synchronization. I've tried to pare these down in to a
> set of questions that may help clarify concerns of users that utilize
> Ubuntu as a kvm guest. I've also done some (rough) experiments to
> answer some of the basic questions.
> 
> My goal is for these to clarify [0] and be eventually update the wiki.
> 
> = Questions =
> 
> * Why precisely does Ubuntu recommend _not_ to use NTP in KVM guests [0]?
>     Some distros and people [9][2][5] recommend using NTP in their
> guests, as well as the KVM FAQ [1]. If kvm-clock accomplishes proper
> clock synchronization, is NTP just a waste of time, or could it
> actually cause bugs and additional problems? Would NTP fail because of
> system time drift or other issues [5]?
> 
> * Does kvm-clock propagate the host system time to guests?
>     Yes, it exposes the host system time as shown in the below
> experiments. Does this also expose UTC corrections such as leap
> seconds [5]?
> 
> * How can Guest VM's update their system time from hwclock?
>    Executing "hwclock --hctosys" updates the system time, but perhaps
> there is a better way?
> 
> * What happens when a system suspends and resumes?
>    It looks like the system still maintains the proper timing.
> 
> = Experiment =
> 
> * Assuming you have a clean KVM guest setup.
>   For my experiment I ran a 3.8.0-19 kernel in an raring amd64 server
> install.
> 
> * For each experiment record the following in the guest:
>   dmesg | grep "setting system clock"
>   echo "date +%s output = " $(date +%s)
>   hwclock --debug | grep "Hw clock"
>   clockdiff 192.168.122.1 #or whatever the host machine IP is
> 
> * Compare to time when you start the vm and whenever you run the script.
> 
> 1) Boot guest and record output, it should be somewhat close in time.
>   * Host before boot: 1366742263
>   [    0.528148] rtc_cmos 00:00: setting system clock to 2013-04-23
> 18:37:48 UTC (1366742268)
> 
>   * Host when executing script: 1366742312
>   date +%s output = 1366742313
>   Hw clock time : 2013/04/23 18:38:33 = 1366742313 seconds since 1969
>   host=192.168.122.1 rtt=750(187)ms/0ms delta=-40ms/-40ms Tue Apr 23
> 13:38:33 2013
> 
>   So this matches expectations.
> 
> 2) Change host system time and record output afterwards.
>   * Host when executing script: 924374443
>   date +%s output = 1366742454
>   Hw clock time : 1999/04/17 18:40:44 = 924374444 seconds since 1969
>   host=192.168.122.1 rtt=750(187)ms/0ms delta=-10513ms/-10513ms Tue
> Apr 23 13:40:54 2013
> 
>   Looks like hwclock is correct, but 'date' doesn't reflect the
> changed time.
>   Running 'sudo hwclock --hctosys' fixes this:
>   $ sudo hwclock --hctosys
>   ubuntu at ubuntu:~$ echo "date output = " $(date +%s)
>   date output =  924374611
> 
> 2) Set host system time to something bogus, boot and check output.
>   * Host before boot: 86415
>   [    0.581315] rtc_cmos 00:00: setting system clock to 1970-01-02
> 00:00:20 UTC (86420)
> 
>   * Host when executing script:
>   date +%s output = 1366742803
>   Hw clock time : 1970/01/02 00:01:03 = 86463 seconds since 1969
>   host=192.168.122.1 rtt=562(280)ms/0ms delta=18859219ms/18859219ms
> Tue Apr 23 13:46:45 2013
> 
>   So again looks like hwclock is correct, but system time isn't
> updated from hwclock on boot.
> 
> 4) Boot machine and record data. Then suspend machine for X minutes,
> resume and record data afterwards.
>   Paused the VM for about 4m.
>   ubuntu at ubuntu:~$ date +%s
>   1366742933
>   ubuntu at ubuntu:~$ date +%s
>   1366743185
>   This delta matches the pause of 4m, so the clock looks proper.
> 
> Below is a list of various references I looked at to help answer these
> questions.
> Thanks,
> --chris j arges
> 
> [0]
> https://help.ubuntu.com/community/KVM/FAQ#Should_ntp_be_used_for_time_synchronisation.3F
> [1]
> http://www.linux-kvm.org/page/FAQ#I.27m_experiencing_timer_drift_issues_in_my_VM_guests.2C_what_to_do.3F
> [2] http://s19n.net/articles/2011/kvm_clock.html
> [3]
> http://rogierm.redbee.nl/blog/2011/11/07/time-synchronisation-in-kvm-guest-to-ntp-or-not-to-ntp/
> [4] https://rwmj.wordpress.com/tag/kvmclock/
> [5]
> http://thread.gmane.org/gmane.comp.emulators.kvm.devel/96835/focus=96981
> [6]
> http://www.mjmwired.net/kernel/Documentation/virtual/kvm/timekeeping.txt
> [7] https://lkml.org/lkml/2010/4/15/353
> [8] http://tldp.org/HOWTO/Clock-2.html
> [9]
> https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/chap-Virtualization_Host_Configuration_and_Guest_Installation_Guide-KVM_guest_timing_management.html

Hi Chris,

I'm not really "the expert" here. As such, I"m not going to weigh in on
the specific questions. I reviewed the links (thanks for doing the dig
for them.) The most relevant seemed to be Marcelo Tosatti's [5] wherein
he says that you need to run ntp in host and guest.

I'd speculate that at one point in time (recall that we've been shipping
kvm longer than anyone) it may have been prudent to only run ntp in the
host side, but things can change.

Also, if you are running a set of interdependent services (say Juju
charms), you likely want reliable time WITHIN the services so you'd
definitely want to be able to run ntp therein. So, if you CAN run ntp
within the VM, we should certainly allow it.

I'd also agree that ideally you'd want to to have the master be the host
machine. However, in a cloud environment, that's not going to be trivial
as you likely have no idea what host you are running on.

That's my $0.02.