[0.18][#109169][merge] add lock debugging, remove unnecessary lock peeks

[Martin Pool]

Here is an updated version in response to feedback from John and
Robert.  The scope has broadened.

 * Add -Dlock trace support
 * Fix https://bugs.launchpad.net/bzr/+bug/118715 that causes many
directories to be left behind when we fail to take a lock
 * Add a Transport simulator for servers that do rename(a,b) as
rename(a, a/b), as bazaar.launchpad.net once did, and a test that we
handle this correctly; also make unlocking more robust in this case.
 * Some general refactoring of LockDir
 * assertContainsRe gives a better failure message
 * tests for locking by token

Because this fixes a high-severity bug I'd like to land it for 0.18.

Also a question: we currently have LockDir.wait, which waits for the
object to be unlocked but does not take the lock.  This does not seem
very useful, because there is no guarantee that it is still unlocked
by the time you do anything with it.  I think I added it for the test
suite, which is the only place where it's called.  Any objections to
removing it in a later patch?

-- 
Martin