Fixing backup and restore

[Curtis Hovey-Canonical]

The backup and restore test has passed 1 out of a 500 runs. I have
never succeeded in using backup and restore. This feature is broken,
and because it is always broken, new bugs have been introduced.

The initial bug caused by a race condition when restore is fixed. I
think there are now 3 other bugs represented in these two bug reports:

    * juju-backup command fails against trusty bootstrap node
      https://bugs.launchpad.net/juju-core/+bug/1305780
    * backup restore timeout restarting agent
      https://bugs.launchpad.net/juju-core/+bug/1304116

I see these issue testing on my own machines and in CI:

* backup assumes mongodb-client is installed. This isn't the case for
trusty because it using juju-mongodb. The client is installed, but the
bash script that is backup assumes mongodump is in the path. on
trusty, we guarantee /usr/lib/juju/bin/mongodump.

* backup can kill the state-server. Well done. If I had gotten the
tarball back from the backup, I could have immediately restored it.
When backup cripples the state server, it is then impossible to juju
scp the file to my local machine. Maybe the issue is simply that the
backup portion of the script exits before juju and mongodb are backup?
Maybe backup needs to be aware of HA changes to make a safe backup?

* I sometimes get a backup file. I cannot restore though. Sometimes
restore states is succeeded, but juju status never works afterwards.
Restore doesn't succeed most of the time, it reports it cannot get the
public address. HA may be  a factor in this since this error appeared
about the time HA was being added to the code.


-- 
Curtis Hovey
Canonical Cloud Development and Operations
http://launchpad.net/~sinzui