[time-nuts] Standards for units

[Tom Van Baak]

> >1/meter/39.37 inches = 0.025400051 meters/inch ...
 
> The interesting thing is that this result is off by  2.000004000 ppm. 
> That's a rather precise error!
> 
> The calculator I used is here:
> http://homepage.ntlworld.com/r.howitt/Calculator.htm

David,

There are two ways to compute relative error, depending on
which one you call the reference.

We start with 39.37 * 25.4 = 999.998 (exactly) and note that
it is 0.002 (exactly) away from 1000.

You can compute the relative error two ways:
1) 0.002 / 1000.0 = 2 ppm (exactly), or
2) 0.002 / 999.998 = your 2.000004000 ppm number.

I'd guess convention is to use the first number, not the second.

For extra credit, note your number is actually this cool-looking
fraction* whose pattern you will immediately recognize.

2.000004000008000016000032000064000128000256000512001024...

/tvb
http://www.LeapSecond.com

* calculated with: echo "80k .002 999.998 / 10 6 ^ * p" | dc