Parametrization being a bit too broad

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've been wondering if some of our parametrization cases are a bit too
broad. We often tend to permute all test cases by say all Repository
implementations. Often, however, many repository implementations share
the same functionality that we are trying to test.

The one I just came across was "bzr selftest -s bt.test_inv". Which
applies 26 different scenarios for each test.

> 167ms test_replaced_at_new_path(Inventory)
> 174ms test_replaced_at_new_path(RepositoryFormatKnitPack1)
> 163ms test_replaced_at_new_path(RepositoryFormat2aSubtree)
> 165ms test_replaced_at_new_path(RepositoryFormatKnitPack5RichRootBroken)
> 149ms test_replaced_at_new_path(RepositoryFormatKnitPack4)
> 128ms test_replaced_at_new_path(RepositoryFormat7)
> 176ms test_replaced_at_new_path(RepositoryFormatKnitPack5RichRoot)
> 150ms test_replaced_at_new_path(RepositoryFormatKnitPack5)
> 142ms test_replaced_at_new_path(RepositoryFormatKnit4)
> 152ms test_replaced_at_new_path(RepositoryFormatKnitPack6)
> 160ms test_replaced_at_new_path(RepositoryFormat2a)
> 179ms test_replaced_at_new_path(RepositoryFormatKnitPack6RichRoot)
> 155ms test_replaced_at_new_path(RepositoryFormatPackDevelopment2Subtree)
> 134ms test_replaced_at_new_path(RepositoryFormatKnit1)
> 136ms test_replaced_at_new_path(RepositoryFormatKnit3)
> 152ms test_replaced_at_new_path(RepositoryFormatKnitPack3)
> 261ms test_replaced_at_new_path(WorkingTreeFormat4.update_basis_by_delta)
> 256ms test_replaced_at_new_path(WorkingTreeFormat4.apply_inventory_delta)
> 260ms test_replaced_at_new_path(WorkingTreeFormat5.update_basis_by_delta)
> 251ms test_replaced_at_new_path(WorkingTreeFormat5.apply_inventory_delta)
> 265ms test_replaced_at_new_path(WorkingTreeFormat6.update_basis_by_delta)
> 286ms test_replaced_at_new_path(WorkingTreeFormat6.apply_inventory_delta)
> 294ms test_replaced_at_new_path(WorkingTreeFormat3.update_basis_by_delta)
> 229ms test_replaced_at_new_path(WorkingTreeFormat3.apply_inventory_delta)
> 138ms test_replaced_at_new_path(WorkingTreeFormat2.update_basis_by_delta)
> 102ms test_replaced_at_new_path(WorkingTreeFormat2.apply_inventory_delta)

Note that these aren't particularly fast tests. Especially the WT ones,
which have to create a WT, populate it, and then apply the delta.

However, WT4,5,6 all share the same DirState update_basis_by_delta
implementation. 2&3 also do, but I don't think we really care about them
anymore.

Similarly, all the KnitPack repositories share an implementation. As
does 2aSubTree,Dev2Subtree, and 2a.

By my count, we really only have 7 or so actual inventory delta
implementations, but we are running the test 26 times.

I like that new implementations automatically get run against the tests.
I'm just wondering if there is a better way of doing that, which doesn't
require us to repeat the test for the same implementations 3 times.

Thoughts?

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk3TmNgACgkQJdeBCYSNAAMm+ACeKnSMhFcT8m15nASZ/dr6jWHb
uYkAoMqGczXgpw297gzWV6/SQDqG/QIX
=91zy
-----END PGP SIGNATURE-----