[time-nuts] flaot representation of phase measurements (was Re: time-nuts Digest, Vol 33, Issue 29)

[Christopher Hoover]

> > Incidentally, I've found that it is important to be quite careful with
> your
> > floating point handling when representing a quantity of seconds with
> > picosecond resolution in an IEEE 64-bit float.
> 
> Can you give me an example of where you'd run into this?
> 
> I ask because typically I think one normalizes incoming
> data so it doesn't matter if the raw values are seconds or
> picoseconds, and thus you get the same 54-bit precision
> either way, no?
> 
> /tvb

There are 52 represented bits of significand (mantissa) in an IEEE 754
double, with one bit implied, for a total of 53 bits.

But the exact number of bits isn't relevant -- no matter how many bits are
used in the significand, given a particular base, there are some rational
values that cannot be represented exactly.   IEEE 754 floats use base 2.
The exact (ha ha!) restriction is that the denominator has to be a power of
2.

As 1 is 2^0, representing integer quantities within a certain (and useful)
range (but well short of the minimum and maximum values) happens exactly in
the IEEE floating point representation.

The implication is that a double quantity of picoseconds may be exact,
whereas the equivalent quantity represented as a double of whole +
fractional seconds may not be, because 1E-12 is not a power of 2.

Of course, if your quantity of integer picoseconds gets too large you will
run into the other end of inexactness.

But, let's get back to the 53132.  Here's a real example where the exactness
issue is evident (assuming Outlook doesn't mangle the lines too badly):

ch at armstrong:~/src/freqstd$ cat /tmp/eg
  1176340715    0.300,072,693,8 s
  1176340716    0.300,072,699,5 s
  1176340717    0.300,072,700,5 s
  1176340718    0.300,072,706,0 s
  1176340719    0.300,072,705,5 s
  1176340720    0.300,072,710,5 s
  1176340721    0.300,072,710,4 s
  1176340722    0.300,072,714,4 s
  1176340723    0.300,072,685,7 s
  1176340724    0.300,072,684,3 s
ch at armstrong:~/src/freqstd$ ./hp5313x-r116.pl < /tmp/eg
  1176340715    3.0007269380000001e-01 
  1176340716    3.0007269949999998e-01 
  1176340717    3.0007270050000001e-01 
  1176340718    3.0007270600000002e-01 
  1176340719    3.0007270549999998e-01 
  1176340720    3.0007271050000001e-01 
  1176340721    3.0007271040000000e-01 
  1176340722    3.0007271440000000e-01 
  1176340723    3.0007268570000001e-01 
  1176340724    3.0007268430000000e-01 
ch at armstrong:~/src/freqstd$ ./hp5313x.pl < /tmp/eg
  1176340715    0.3000726938
  1176340716    0.3000726995
  1176340717    0.3000727005
  1176340718    0.300072706
  1176340719    0.3000727055
  1176340720    0.3000727105
  1176340721    0.3000727104
  1176340722    0.3000727144
  1176340723    0.3000726857
  1176340724    0.3000726843
ch at armstrong:~/src/freqstd$ 

/tmp/eg is the raw data from a 51312 serial port prefixed with time_t
timestamps.

./hp5313x-r116.pl is an earlier version of my conversion script that reads
the TI values as IEEE doubles, scales them according to the suffix using
IEEE double arithmetic, and prints them likewise.  As you can see the values
aren't exact -- and this is multiplying by (an exactly represented) 1.0.

./hp5313x.pl uses big floats.  The values are read exactly, multiply
exactly, and printed exactly.

Certainly we can argue if/when this matters -- it does not in many cases,
e.g., calculating adev(*).    I particularly care here as I archive this
(lightly) processed data; I want these simple transforms I do early on be
correct and exact.

-ch

(*) It would also be bloody slow not to use IEEE doubles and hardware
support.