[time-nuts] Linux TSC clocksource on multi-core systems

[Laszlo Hanyecz]

The problem isn't so much NTP, it's that the kernel can't use the TSC as the clock source, so it falls back to something like the HPET or ACPI timer which are orders of magnitude less resolution than the TSC.

Here is the Atom D510 that shows the synchronization problem.  Booting with 'nosmp' allows the TSC to be used.  It's a single board computer, it has one physical CPU, and has no way to disable the power saving stuff in the BIOS, but it advertises a constant counter (constant_tsc in cpuinfo).  The problem is just that the counters aren't in sync, I think.

[    0.168557] smpboot cpu 2: start_ip = 99000
[    0.260015] TSC synchronization [CPU#0 -> CPU#2]:
[    0.260015] Measured 50 cycles TSC warp between CPUs, turning off TSC clock.
[    0.260015] Marking TSC unstable due to check_tsc_sync_source failed


I wonder if it's possible to sync the TSCs between the cpus instead of just abandoning it.  It takes time to measure the counter and to set it, so maybe it could be done by iteratively making small changes and comparing the counters, and trying to eliminate the effects of the read/write delay.  Not sure if it's possible to completely sync them but maybe they could be brought really close?

This system is just an example, but I think these TSC problems are pretty common.  I had an AMD CPU based desktop computer that was dual core and the only way to play games on it was to set the game process affinity to a single core, due to the TSC sync issue.  Without that, the game ran really fast but in a jerky way.  AMD had a workaround for it.. it was a program that ran as a service and I think it synced the counters many times a second, but I'm not sure how close it really got.  I guess it could have been way off but still have been good enough for a 60Hz game.

I have some systems with multiple physical CPUs that do seem to have properly synchronized TSC counters, at least linux doesn't complain and it seems to work, so it seems like it's possible.

Thanks,
Laszlo


On Apr 26, 2014, at 10:02 PM, Michael Tharp <gxti at partiallystapled.com> wrote:

> On 04/26/2014 02:27 PM, Laszlo Hanyecz wrote:
>> It's fine to disable the additional cores/cpus on a dedicated NTP machine, but I wonder if there is a solution that allows both the TSC and all the cores to be used at the same time.  Is it even possible to completely sync the counters across CPUs (not just get close)?
> 
> You could try pinning ntpd to a single CPU using the "taskset" command. It wouldn't give all the applications the benefit of a perfectly synchronized clock, but if you just want ntpd to be happy then it ought to work as well as turning SMP off.
> 
> _______________________________________________
> time-nuts mailing list -- time-nuts at febo.com
> To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
> and follow the instructions there.