Slow tests

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Pool wrote:
> Thanks, that's cool to get some patches for the slow tests already.
> 
> In the hopes of provoking more, here are the slowest classes and the
> slowest modules, and the script to get them, with
> 
> subunit-ls --no-passthrough --times < timings|python sum-modules.py >
> per-module-times
> 
> They're pretty similar, indicating that we do indeed have single large
> classes for many modules.
> 
> It's interesting that bundles are the slowest non-parameterized (?)
> tests.  It doesn't seem like something that should be inherently very
> slow to test.  transports are probably testing many combinations.
> commit builder doesn't seem like it should be the slowest, though
> that's partly because it's testing 8 different format combinations.
> 

Bundles are implementation tests. They are done via inheritance, though.

Also, the tests written there were written by *me* about... 3 years ago
when I first started hacking on the bzr code. That was when I wrote the
initial plugin for 'changesets'. The tests could be *much* leaner. When
I was writing them, it was very much a 'test everything in every test'
sort of thought.

I know Aaron updated them a bit, so maybe they aren't as bad as they
once were, but they're pretty bad.


>  193.547 bzrlib.tests.per_repository.test_commit_builder
>  110.648 bzrlib.tests.per_transport
>  108.599 bzrlib.tests.per_repository.test_repository
>   79.307 bzrlib.tests.per_bzrdir.test_bzrdir
>   55.043 bzrlib.tests.per_branch.test_stacking
>   45.405 bzrlib.tests.per_repository.test_fetch
>   42.119 bzrlib.tests.per_branch.test_branch
>   40.250 bzrlib.tests.per_repository.test_fileid_involved
>   38.123 bzrlib.tests.per_repository.test_reconcile
>   37.214 bzrlib.tests.test_bundle
>   31.390 bzrlib.tests.per_intertree.test_compare
>   29.309 bzrlib.tests.per_workingtree.test_eol_conversion
>   27.890 bzrlib.tests.per_interrepository.test_fetch
>   25.482 bzrlib.tests.test_source
>   25.264 bzrlib.tests.per_repository.test_check_reconcile
>   22.421 bzrlib.tests.per_repository.test_write_group
>   21.043 bzrlib.tests.blackbox.test_non_ascii
>   20.134 bzrlib.tests.per_branch.test_push
>   18.882 bzrlib.tests.blackbox.test_too_much
>   17.967 bzrlib.tests.per_workingtree.test_workingtree

^- I would actually be interested in seeing a 'time per test' sort of
view. As if you are running 2x the number of tests, it is probably
reasonable to take ~2x longer.

Also interesting is if you brought it up even higher, and saw how much
time is 'per_repository' versus everything else.

> 
> 
>  193.547 bzrlib.tests.per_repository.test_commit_builder.TestCommitBuilder
>  110.648 bzrlib.tests.per_transport.TransportTests
>   81.475 bzrlib.tests.per_repository.test_repository.TestRepository
>   74.867 bzrlib.tests.per_bzrdir.test_bzrdir.TestBzrDir
>   44.186 bzrlib.tests.per_repository.test_fetch.TestFetchSameRepository
>   40.442 bzrlib.tests.per_branch.test_stacking.TestStacking
>   30.960 bzrlib.tests.per_repository.test_fileid_involved.TestFileIdInvolved
>   29.309 bzrlib.tests.per_workingtree.test_eol_conversion.TestEolConversion
>   27.077 bzrlib.tests.per_interrepository.test_fetch.TestInterRepository
>   25.476 bzrlib.tests.test_source.TestSource
>   25.264 bzrlib.tests.per_repository.test_check_reconcile.TestFileParentReconciliation
>   25.224 bzrlib.tests.per_repository.test_reconcile.TestsNeedingReweave
>   24.460 bzrlib.tests.per_intertree.test_compare.TestIterChanges
>   21.350 bzrlib.tests.per_branch.test_branch.TestBranch
>   21.043 bzrlib.tests.blackbox.test_non_ascii.TestNonAscii
>   19.585 bzrlib.tests.per_repository.test_repository.TestCaseWithComplexRepository
>   17.791 bzrlib.tests.per_workingtree.test_workingtree.TestWorkingTree
>   17.298 bzrlib.tests.per_repository_reference.test_get_record_stream.TestGetRecordStream
>   15.954 bzrlib.tests.per_repository_reference.test_fetch.TestFetch
>   15.658 bzrlib.tests.per_branch.test_push.TestPush

I think the per_transport could also be quite a bit leaner. Especially
if we finally got rid of the 'multi' apis... They aren't used 'in anger'
anymore. And I don't think any actually got optimized to be any faster
than a 'for path, content in data: put(path, content)' (ie, looping over
the single form.)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkq0OcMACgkQJdeBCYSNAANRIACgy3xgV2Sj7OcQH/KUmPuVmcHL
exYAoKxF49KVUi7m8HQwnD8eQl/88yAC
=1z1R
-----END PGP SIGNATURE-----