[time-nuts] HP 5370B

[Mike S]

At 08:02 PM 5/10/2008, GandalfG8 at aol.com wrote...
>  This has nothing to do with fraudulently claiming compliance, it 
> just
>started as a well reasoned discussion of tolerances and the 
>implications  thereof.

There can be no other conclusion from the statements made.

"Most microwave network analyzers have amplitude resolution of 0.01dB, 
while their accuracy is just around 1dB in most cases...I have had to 
argue too many times that a piece of equipment with a 2dB p-p 
requirement on flatness was just fine when it measured 2.01dB on the HP 
network analyzer. I would not have gotten in that argument if the data 
had been 1.99dB."

"I have never had a piece of equipment rejected because a reading was 
1.99 for a spec of 2 max"

The statements were made with regard to instrument 
resolution/accuracy/precision. Clearly, measuring 2.01 (or 1.99) on an 
instrument with an accuracy of 1 does not allow compliance with a 
specification of 2 to be met.

It was only later that the red herring of significant digits was 
brought up. A specification of "2" is ambiguous in that regard anyway.