Network glitches costing 15 minutes a pop

[Mark Hammond]

> On Fri, 2008-08-15 at 10:46 +1000, Mark Hammond wrote:
> > Fairly regularly, I notice pulling a small number of updates from
> > launchpad can take a very long time - 30 minutes isn't uncommon.
> > Looking in the logs, I see:
> ..
> > It seems to me that little network glitches aren't particularly
> > unexpected - but waiting 15 minutes when it happens isn't that
> > friendly.  Is this something specific to Windows?  Specific to
> > pycurl?
> > Any suggestions about what we can do to make such errors have less of
> > an impact?
> 
> I'm smelling a TCP timeout; 903 seconds is just too regular a number.

Me too - but who is setting that timeout? :)

> What is error 18 - lets determine if its pycurl or a windows socket
> error; and see what type it is.

Good question:  errno.h on windows has:

#define EXDEV           18

which isn't much help, even with google.  Windows itself defines error 18 as
ERROR_NO_MORE_FILES and has the english text of 'There are no more
files.\r\n' - but looking at the process with 'process explorer' shows only
1 socket at a time being used, and no evidence of runaway handle counts - so
I'm not sure that is relevant either.

Ahh - here we go:  libcurl error codes:
| CURLE_PARTIAL_FILE (18)
| A file transfer was shorter or larger than expected. This happens when the
server 
| first reports an expected transfer size, and then delivers data that
doesn't match 
| the previously given size.

So that is just reflecting what the log is already telling us.

> I would guess at a broken intercepting proxy if I had to guess. But
> lets not guess.

There is no proxy on my LAN, and I'm pretty sure I saw the same problem when
I was in Sydney with you, Martin etc - I was trying to demonstrate the
slowness to Martin, but then couldn't reproduce it.  I can't say for sure,
but I'm confident the problem was the same one - and the networks were
completely unrelated.

Thanks,

Mark