selftest results for lp:bzr revno.4804

[Vincent Ladeuil]

Some data for you from other OSes:

- hanging *can* occur, using --parallel=fork helps when the hang
  is caused by leaks,

- slowdowns have been observed due to *socket* leaks. Roughly
  speaking, some test servers use select() on a global pool of
  sockets, the bigger the pool, the slower the select

- leaks aren't reported when using --parallel=fork

So please, when reporting failures and timings on windows, specify:

- whether you used --parallel=fork or not,
- how many CPUs are used (if applicable), 
- how many tests leaks were displayed at the end (if applicable).

In summary: depending on the OS, passing the full test suite may
*require* to use --parallel=fork and 4 CPUs so that the whole
test suite is split in parts small enough to avoid the
leak-related bugs.

4 is the magic number such as:
- it's greater than 2 where my OSX slaves fails to pass,
- is smaller than 8 where my main desktop pass
- is equal to the number of CPUs most of my VMs use (hardy,
  jaunty, karmic, freebsd, gentoo) and pass

>>>>> "Gordon" == Gordon Tyler <gordon at doxxx.net> writes:

    Gordon> John Arbash Meinel wrote:
    >> Alexander Belchenko wrote:
    >>> IIUC latest improvements for selftest on win32 from John has landed.
    >>> I've tried to run tests on lp:bzr revno.4804.
    >> 
    >>> selftest running for almost 3 hours on my Windows XP and finished with
    >>> result:
    >> 
    >>> Ran 23081 tests in 9306.360s
    >> 
    >>> FAILED (failures=13, errors=500, known_failure_count=27)

How many leaks ? This can be very important as errors and
failures can often be caused by leaks.

    >> 
    >> I think it would be interesting to run this again with the most recent
    >> bzr.dev. I also would like to note a few things:

    Gordon> I just ran this with bzr.dev r4852 and I had 1 failure and 2 errors
    Gordon> running the full test. It took about 1h20m.

Using --parallel=fork right ? How many CPUs ?

    >> 1) Do you have a single CPU? I tried running on EC2, and it seems that
    >> if I ran on a single threaded machine, I had problems with SFTP +
    >> file.close() issues. (by default it runs asynchronously, and it seems
    >> that on a single CPU, that means that it *doesn't* run before we try to
    >> rename a directory, etc.)

Is that still true ? You seemed to have found a fix in another thread ?

    Gordon> Intel Core i7 -- quad core, hyperthreaded, so it
    Gordon> looks like 8 CPUs.

Nice babe :) 

How much RAM ? Do you observe swap or not ? Is your /tmp on disk
or in memory ?

   Vincent