A cautionary tale about "go test -race -cover"

[Dimiter Naydenov]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey all,

tl;dr; When running $ go test -race -cover with go < 1.3, and you get
data races, try adding "-covermode=atomic" as well to see if they are
caused by the go coverage tool itself. In go 1.3+ -covermode defaults
to atomic, so yet another reason to really move quickly to 1.5 across
the board ASAP.

I've discovered the hard way the go race detector doesn't play well
with the go coverage tool in golang versions prior to 1.3. With -cover
(and/or -coverprofile=filename, which assumes -cover), the code
instrumentation generated by the coverage tool uses global variables
to track which lines are covered and how many times. This causes the
race detector to report data races in confusing locations (e.g. I had
a whole bunch of them in the provider/ec2 pacakge), which are not
happening without -cover enabled. I've discovered the issue very
easily by adding -test.work argument to "go test". This causes "go
test" to print the temporary work dir used and also keep it after "go
test" completes. Then it's a simple matter of inspecting the work dir
and the actual source files in there, which include race and coverage
instrumentation code added. The DATA RACE output includes paths in
that work dir and line numbers exactly matching the code generated by
- -cover.

Apparently, adding -covermode=atomic along with -cover and -race,
works around the problem in both golang versions < 1.3 and 1.3+, by
using atomic inc ops rather than global non-goroutine-safe vars.

An interesting article describing the issue with examples:
http://herman.asia/running-the-go-race-detector-with-cover

Hopefully this will be helpful to somebody else and save them a few
wasted hours trying to figure out why the result of "go test -race
- -cover" has data races reported, while "go test -race" or "go test
- -cover" for the same tests shows no races at all.

- -- 
Dimiter Naydenov <dimiter.naydenov at canonical.com>
Juju Core Sapphire team <http://juju.ubuntu.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJVp4kCAAoJENzxV2TbLzHwyJQIALb3MuDz9qSuLwhNNL7DfIjq
DsEV14F82qDbdXcjS42f8I3ZIaq3ii8DQZWyR3dNoTtMOhc4thxIVs6LzHdzUR88
GxFLWZeSgsRzeGj0OVvVGeelwIkqUPORt2Q4m2uziPgVkiczniVwkM9FAC1kdu21
l54ibsIu1TsPwDUQzp6LPSEzP7zhv3pB9h8Qhq4sxoqSUjtQXt6ysgKn88Br/Cjl
7EBkcI0sDkm6bglklztNBh8fQqLUXLHW+cUXhdr/7psnhsAaGXHDEC3u3hPi16X+
Mc0yOrsyOHBNdsTV3J8aWiOI9AJFZWjwEWo1LbUc6vL1znX40LF2Uxq7tTuTJ6g=
=DXZL
-----END PGP SIGNATURE-----