test isolation: default ignores & repository acquisition

[Vincent Ladeuil]

>>>>> "martin" == Martin Pool <mbp at canonical.com> writes:

    martin> 2009/9/18 Vincent Ladeuil <v.ladeuil+lp at free.fr>:
    >> Exactly what I pointed in an another mail, the tests shouldn't
    >> start with a local setup if that could be factored out at the
    >> class level, thanks for demonstrating my point.
    >> 
    >> Having setUp() methods makes smaller and more readable
    >> tests. Smaller tests provides better defect localization, by
    >> design.

    martin> I think separating out common code is (of course) good, but I do not
    martin> feel very convinced that putting it in setUp(), as opposed to an
    martin> explicitly called make*() method is a good idea.  

    martin> It is less explicit,

I disagree :)

The intent is made more explicit in the class *name*.

    martin> it doesn't seem much shorter,

    martin> you can have bugs with not calling the superclass
    martin> setUp(),

Come on ! Tried it lately ? You're caught every time. Robert and
Andrew set up the trap months ago.

    martin> and addCleanup is better than putting cleanup it
    martin> tearDown.

Nothing wrong with calling addCleanup in setUp !

    martin> In other words I prefer

    martin>   def make_branch_with_history(self):
    martin>     branch = self.make_branch()
    martin>     ...

    martin>   def test_some_thing(self):
    martin>     branch = self.make_branch_with_history()
    martin>     ...

    martin> to

    martin>   def setUp(self):
    martin>     super(SillyPythonTest, self).setUp()
    martin>     self.branch = self.make_branch()
    martin>     ...

    martin>   def test_some_thing(self):
    martin>     check_something(self.branch)

I understand that and I can show you many examples where I remove
similar lines at the beginning of *all* tests in some classes. 

I don't want to forbid you to do that occasionally, but doing it
systematically is clearly wrong.

I can show you many examples where doing that were leading to
test classes where not all tests were sharing the same setup (and
that make them harder to maintain).

Can you show me examples where your approach give benefits, and
which ones[1] ?

A test can live in many places, some are better than others, test
classes are there to give the best possible place to a test by
providing helpers to make the test easier to write (and more
importantly easier to read), not just group them semi-randomly.

Again I don't ask to have perfect classes for each test, but
clearly we can do better than we do now, and I think having more
setUp() methods helps to make better tests. 

When refactoring some really old tests, you find the copy/paste
really quick.

I don't ask for more code sharing just on theoretical bases, I've
deleted a lot of cruft in real life :)

      Vincent

[1]: I think the discussion is unbalanced between ease of write,
     ease of read and ease of maintain pros/cons and I really
     like to understand where we all stand.