Another timing dependent test

[Martin Pool]

On 4/14/07, John Arbash Meinel <john at arbash-meinel.com> wrote:
> I'm trying to ferret out some of the random-failures for PQM tests. Most
> of them are because of dependency on time.time() in one way or another.
> I sent in a patch for 2 of them earlier. But I have another one I'm not
> sure what to do with:
>
>
> class SFTPLatencyKnob(TestCaseWithSFTPServer):
>     """Test that the testing SFTPServer's latency knob works."""
> ...
>
>     def test_default(self):
>         # This test is potentially brittle: under extremely high machine
> load
>         # it could fail, but that is quite unlikely
>
> I found it... interesting that the test itself understood it could fail
> but was "quite unlikely".

"Side effects include headaches, nausea... sudden death"

At the moment this is testing that you can get a transport in less
than 500ms, which while desirable is not really a functional
requirement or something we can insist on.  It might be a reasonable
thing to have while adding the latency knob but it's not helping any
more.

So I would ask, what kind of regression is this test trying to catch?
Maybe that the server has a latency delay by default?  In that case I
would just assert that the test's server has add_latency == 0.

> I know this really wants to test that the latency knob is making things
> slower, and it doesn't make much sense to test that it is slow with the
> knob turned up without testing that it isn't slow without the knob.

I don't think that necessarily follows...

-- 
Martin