s3.AttemptStrategy and the juju-core test suite

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I managed to track down where a lot of the time is spent during
'launchpad.net/juju-core/environs/ec2' tests.

Specifically, all the time appears to be spent looping in
s3.GetReader() calls, which are failing with "The specified bucket
does not exist" and the default timeout logic of retry-up-to 3.0s.

We specifically spin on the default retry attempts to:

1) Load 'provider-state' which doesn't exist
2) Look for Mongo tools in the private bucket (doesn't exist)
3) Look for Mongo tools in the public bucket (doesn't exist)

AttemptStrategy stops trying when cur+sleep > max, so it doesn't
actually wait the full 3s. It waits 3-200ms = 2.8s. However, that
means that we spend 8.4s sleeping during every "Bootstrap()" call.
Unfortunately, because of how the go profiler works, it doesn't tell
you what stack was asking to sleep, so I added a helper for getting
the actual time spent in code.

Now, I can just expose the 's3.SetAttemptStrategy' code (which is only
exposed for the s3 test suite) and use it in our local tests. This
drops the time for environs/ec2 from 1m40s down to 22s for me.

However, I feel it might be a sign of a deeper issue.

a) Why do we have 2 implementations of AttemptStrategy
(juju-core/trivial, goamz/ams) that are slightly different?

b) Should the attempt strategy actually just be a parameter to the
s3/amz/ec2/goose/etc objects (rather than global state we have to
override)? We could allow 'nil' to be passed, and then have it use the
default timeouts we already have.

c) Should s3 itself be doing the retry rather than exposing the retry
up to the higher level? [A gui using the library may want to give a
quick update that the bucket doesn't exist even while it is doing a retry]

Thoughts?

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlE+x+UACgkQJdeBCYSNAANaFgCdG8hI5JNG1tTm8adSs3iGopcB
jfQAoNcHVekjowHWuY6S5Nz+iX/TvND8
=2mTU
-----END PGP SIGNATURE-----