Wrote a small black-box test suite for bazaar

[Geoff Bache]

Hi all,

I attach this in case anyone thinks it might be useful. I don't mean to
imply that Bazaar is inadequately tested
but I thought this might be a useful addition to its current testing. It
didn't take me long to produce it and
I think you could easily produce a lot more of this kind of test quite
quickly

It makes use of TextTest which you can download from
http://sourceforge.net/project/showfiles.php?group_id=80138
Further information lives at http://www.texttest.org. (I'm currently in the
process of migrating TextTest to use
Bazaar and Launchpad.)

Basic instructions: install PyGTK and tkdiff, unzip the attached test suite
and the TextTest download, cd to the
test suite directory and run texttest.py from wherever you unzipped it.

The basic plan: it runs bazaar from the command line and compares stdout,
stderr, a generated file describing all files
created, edited and deleted, and the bazaar log. Run-dependent entries in
these are filtered away. It also runs everything
on copies of the data which are then removed so you don't have to reset
anything. BZR_HOME, the bazaar repository .bzr,
and the working tree are all test data and vary throughout the structure
independently of one another.

I've run it on bazaar 1.3.1 and 1.5 and the tests work unchanged between
those versions. But the baseline files are easy to "save"
as and when this is needed.

The main points:
1) You don't need to know the code to write tests (I've never looked at the
code)
2) Tests don't depend on the structure of the code and hence don't need
changing when the code is refactored.
3) There are already quite a few blackbox tests that look like

def test_update_standalone_trivial(self):
        self.make_branch_and_tree('.')
        out, err = self.run_bzr('update')
        self.assertEqual('Tree is up to date at revision 0.\n', err)
        self.assertEqual('', out)

This is basically a way to write tests of that form without writing any
code.

Let me know if anyone is interested in trying this out. Am happy to provide
further information if needed.

Regards,
Geoff Bache
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/bazaar/attachments/20080826/c160ffd3/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bazaar.tar.gz
Type: application/x-gzip
Size: 7991 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080826/c160ffd3/attachment-0001.bin