EOD 2013-01-21

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2013-01-22 0:10, Martin Packman wrote:
> Ian asked in the standup if I could land my branch with the 
> secgroup-delete-all tool as he had a use for it the other day.
> 
> However, I didn't want to merge it with no tests - especially as
> it seems just running `go test ./...` doesn't fail if there are
> build problems in a package without a test file. It turns out
> writing tests using our current testservice doubles is pretty
> painful, and it's still not possible to properly cover error
> cases.
> 

Just to note, as I'm setting up tarmac, I'm making the official "test"
policy:
 go build && go test ./...

So build has to pass as well as 'go test'. Which should help us find
those cases where we accidentally have live code that depends on a
variable only defined in test infrastructure, etc.

That should alleviate your immediate concern. (The code is broken but
without tests it is never compiled.)


> I've reproposed the branch with a couple of test cases added, if 
> anyone wants to have a look:
> 
> https://codereview.appspot.com/6948051/
> 
> In particular: * I'd like to write proper unit tests but we're
> stuck with end-to-end testing mostly. * I don't see how to test the
> basics of command line behaviour with gnuflag magic involved.

Is this an issue of how to run a binary (so you can pass it command
line parameters), or is it how to run the data in memory but setting
up the flag content? I think you can just write to the variables,
though you might need to expose functions for a test suite. (in
expose_test.go write a SetFoo() function that the test suite can use
to set the value of the flag variable.)

I'm pretty sure gnuflag just uses pointers so that you have a real
basic type as the flag value, and gnuflag just sets them when the
command line parameters are parsed.

> * There's no way to set up state for novaservice currently, and
> its internals are private.

Is this, "I want 3 instances running, and for them to be doing X", but
not having to go via the public "StartInstance" calls?

> * We have no way make our test services return errors on demand.

This goes along with (1), though there are 2 possible ways of doing
it. Ian has proposed we send stuff in the headers of request
(X-Goose-Fail: ...). Which could work, but I have the feeling we might
find it easier to talk directly to the double (s.Double.FailOnX())
rather than injecting a header at the right time. I'm open to either.

I certainly think we want to be able to set up the double state (you
have this running, you want to fail in this way).

> * The http response code of the delete call tested was wrong in the
> double.

SMOP? It seems like just a missing test.

> * There is a lot of setup required and not much in the way of
> shortcuts.
> 

I do think we want to end up with a "SetUpOpenstack()" sort of
high-level set-up-the-world function. The only reason I can think of
to not set up everything is if it is expensive (in human time) to do
for every test. But these are all in-memory-only things, so it should
be quite cheap. And much better, IMO, to avoid the human overhead of
"ok, what services do I need, and how are they configured, etc".

> Martin
> 

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlD+L2wACgkQJdeBCYSNAAMfnwCgzAiXAVeQ1nhU85fRtYNpJHPX
wl4An33sD1ce9kwZzPTDUOhKOLtLT/WG
=Bmdq
-----END PGP SIGNATURE-----