Another timing dependent test

[Martin Pool]

On 4/17/07, John Arbash Meinel <john at arbash-meinel.com> wrote:
> Well, there are 2 paired tests (atm).
>
> test_default
> test_latency_knob_slows_transport
>
> The former tests that it takes <500ms, and the latter tests that when
> latency is added, it takes longer than 400ms.
>
> Without the former, you have no baseline for the latter test. The latter
> test could succeed just because we are always very slow at opening a
> transport object.

(I guess my post is saying it's ok to have tests that sometimes
falsely pass because of timing effects.  It's certainly less annoying
but on reflection it's not really desirable.)

> I can understand not wanting to test a timing side-effect.  It just
> isn't clear to me how to test this specifically. I realize that
> "test_latency_knob_slows_transport" is a functional (as opposed to unit)
> test. But it seems like a general meaningful test to have...

Well, then I guess as you said before we can hook time.sleep and
observe how many times it is called.  (Or it might be a little cleaner
and have a variable that normally points to time.sleep but that can be
redirected.)

Also:

<lifeless> what we really want to test is that the knob *adds* that much latency
 to requests, not to getting the transport (getting the transport is
defined as something that doesn't cause a connection these days)
<lifeless> we can test that the time to do a single op increases by
*at least* the time the knob claims
 that should be robust against timing sensitivity (or the code is broken)
 it means that we dont need to care about platform details in the test
- they can be encapsulated in the code

-- 
Martin