[RFC] simulating network properties for benchmarks

[Martin Pool]

On 21 Jul 2006, Robert Collins <robertc at robertcollins.net> wrote:
> Heres a prototype of what I was referring to. The decorator in it was
> quick-coded, if the future directions I mention in it are to be done, it
> would deserve unit tests, as high layer tests could no longer test it at
> that point.
> 
> I'm pushing this up to
> sftp://bazaar.launchpad.net/~bzr/bzr/test-sftp-latency now.

I think this is a good approach.  OK, so it may not simulate delay at
the individual-packet level, but I think it'll be pretty close.

> +
> +
> +class SFTPLatencyKnob(TestCaseWithSFTPServer):
> +    """Test that the testing SFTPServer's latency knob works."""
> +
> +    def test_make_transport_slow(self):
> +        # change the latency knob to 100ms. We take about 40ms for a 
> +        # loopback connection ordinarily.
> +        self.get_server().add_latency = 0.1
> +        self.assertConnectionTime(gt)
> +
> +    def assertConnectionTime(self, operator):
> +        start_time = time.time()
> +        transport = self.get_transport()
> +        stop_time = time.time()
> +        self.failUnless(operator(stop_time - start_time, 0.1))
> +
> +    def test_default_fast(self):
> +        self.assertConnectionTime(lt)

Hm, cute.  Not quite sure if that's good-cute or bad-cute, compared to
say

  self.assert_(self.measure_time() > 0.1)

> +class SocketDelay(object):
> +    """A socket decorator to make TCP appear slower.
> +
> +    This changes recv, send, and sendall to add a fixed latency to each 
> +    python call. It will therefore behave differently to the real underlying
> +    TCP stack which may only pay the latency price on each actual roundtrip
> +    rather than each packet initiated. We can come closer to the real
> +    behaviour by only charging latency for each roundtrip we detect - that is
> +    when recv is called, toggle a flag, and insert latency again only when
> +    send has been called in the middle. For now though, showing more latency
> +    than we have is probably acceptable.
> +

It may also help to insert a bandwidth cost proportional to the amount
of data sent, which will tend to reduce the penalty on small reads or
writes without complicating things too much. 

We could spend any amount of time (heh) on getting packet-accurate
delay simulation but I think just sleeping per transmission and per byte
will be enough.  So let's go with something like this, and tune it later
if it looks unrepresentative.

-- 
Martin