[time-nuts] Standards for units
Hal Murray
hmurray at megapathdsl.net
Mon Apr 2 23:55:03 EDT 2007
> My recollection when I posted the earlier letter was 24.5 mm/inch was
> exact by definition, and I recalled that 39.37 inches/meter was also
> exact... a quick calculator check to 6 figures showed that it was
> exact, then I tried to 9 figures, and saw that pesky little
> 0.000000051.
I did things like that for a while, then I multiplied 3937 by 254. The
answer is 999998. So those 2 constants are off by 2 ppm from being an exact
inverse of eachother.
I assume the decimal points/units work out right. I just shifted to integers
to avoid any rounding quirks.
--
These are my opinions, not necessarily my employer's. I hate spam.
More information about the time-nuts
mailing list