<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
On 04/27/2016 09:51 PM, Nate Finch wrote:<br>
<blockquote
cite="mid:CAK=yn+uzD9qOshSMBvSk63TOEczUs1mOUKRG1aM4jvuB2fTR3A@mail.gmail.com"
type="cite">
<div dir="ltr">So, this is exactly why I didn't want to mention
the nature of the test, because we'd get sidetracked. I'll make
another thread to talk about that specific test.
<div><br>
</div>
<div>I do still want to talk about what we can do for unit tests
that take a long time. I think giving developers the option
to skip long tests is handy - getting a reasonable amount of
coverage when you're in the middle of the develop/test/fix
cycle. It would be really useful for when you're making
changes that affect a lot of packages and so you end up having
to run full tests over and over. Of course, running just the
short tests would not give you 100% confidence, but once
you've fixed everything so the short tests pass, *then* you
could do a long run for thorough coverage.</div>
</div>
</blockquote>
<br>
I believe Cheryl has something like this in the works and will be
sending a note out on it soon.<br>
<br>
<blockquote
cite="mid:CAK=yn+uzD9qOshSMBvSk63TOEczUs1mOUKRG1aM4jvuB2fTR3A@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>This is a very low friction way to increase developer
productivity, and something we can implement incrementally.Â
It can also lead to better test coverage over all. If you
write 10 unit tests that complete in milliseconds, but were
thinking about writing a couple longer-running unit tests
that make sure things are working end-to-end, you don't have
the disincentive of "well, this will make everyone's full
test runs 30 seconds longer", since you can always skip them
with -short.</div>
</div>
<div><br>
</div>
<div>The only real negative I see is that it makes it less
painful to write long tests for no reason, which would still
affect landing times.... but hopefully everyone is still aware
of the impact of long-running tests, and will avoid them
whenever possible.</div>
</div>
</blockquote>
<br>
I will gently point out that we were prepared to land a test that
takes ~17s to run without discussion. The motivations are honest and
good, but how many others think the same? This is how our test suite
grows to be unmanageable.<br>
<br>
I also agree with Andrew that the nature of the test should be the
delineating factor. Right now we tend to view everything through the
lens of the Go testing suite; it's a hammer, and everything is a
nail. Moving forward, I think we should try much harder to delineate
between the different types of tests in the so-called <a
href="http://martinfowler.com/bliki/TestPyramid.html">test
pyramid,</a> place like tests with like tests, and then run
classes of tests when and where they're most appropriate.<br>
<br>
This is definitely static analysis and should be run in the same
stage as other static analysis tools.<br>
<br>
-<br>
Katherine<br>
</body>
</html>