[Bug 1793612] Re: node-tap in bionic fails autopkgtests due to timeouts
Mauricio Faria de Oliveira
mfo at canonical.com
Fri Sep 21 10:49:08 UTC 2018
** Description changed:
+ SRU Template:
+
+ [Impact]
+
+ * node-tap autopkgtests intermittently fail due to timeouts
+ depending on the load in the server running the tests.
+
+ * This does not provide reliable test results, misleading
+ the analysis of new code changes to this package, or
+ other packages which this package depends on (as that
+ triggers a re-test for this package).
+
+ * This requires manual effort to re-trigger tests just to
+ verify whether the errors are reproducible/intermittent.
+
+ [Test Case]
+
+ * It's possible to reproduce the failures due to short
+ timeout values by manually specifying shorter timeouts
+ than the defaults, either via TAP_TIMEOUT environment
+ variable or '-t TIMEOUT' parameter, and in setTimeout()
+ calls (e.g., in test/spawn.js);
+ Then run the test suite in either one of these methods:
+
+ * Run tests like 'debian/rules override_dh_auto_test'.
+
+ * Run autopkgtest locally.
+
+ * The autopkgtest server log history shows a pattern of
+ failures due to timeouts, with a different test being
+ hit/stopped (and the following tests planned in the
+ .js file not being run) each time, which can be used
+ as evidence of how the errors look like.
+
+ [Regression Potential]
+
+ * Low. The changes only affect build-time execution,
+ not anything that has runtime changes.
+
+ * The differences in build dependencies between the
+ original build and a rebuild with this patch should
+ not be carried in the package, as nodejs modules
+ should mostly be binary-indepedent/arch-all, thus not make
+ it into the package but rather are sourced at runtime.
+
+ Bug Description:
+
The node-tap package in Bionic usually fails autopkgtests
with several intermittent errors caused by timeouts, that
depend directly on the load of the autopkgtest server.
It's very difficult to reproduce these failures on a local
computer that is lightly-loaded, but it's possible to force
them to happen very similarly to the autupkgtest error logs
by manually decreasing/specifying the timeout values.
The same package in Cosmic passes all autopkgtests, and the
code differences between both releases include 2 commits in
Debian that cause the timeout values to be increased.
So, pull those 2 commits from Debian & Cosmic to Bionic, and
allow the tests to finish properly, so to have more reliable
autopkgtest results.
--
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1793612
Title:
node-tap in bionic fails autopkgtests due to timeouts
Status in node-tap package in Ubuntu:
Fix Released
Status in node-tap source package in Bionic:
New
Bug description:
SRU Template:
[Impact]
* node-tap autopkgtests intermittently fail due to timeouts
depending on the load in the server running the tests.
* This does not provide reliable test results, misleading
the analysis of new code changes to this package, or
other packages which this package depends on (as that
triggers a re-test for this package).
* This requires manual effort to re-trigger tests just to
verify whether the errors are reproducible/intermittent.
[Test Case]
* It's possible to reproduce the failures due to short
timeout values by manually specifying shorter timeouts
than the defaults, either via TAP_TIMEOUT environment
variable or '-t TIMEOUT' parameter, and in setTimeout()
calls (e.g., in test/spawn.js);
Then run the test suite in either one of these methods:
* Run tests like 'debian/rules override_dh_auto_test'.
* Run autopkgtest locally.
* The autopkgtest server log history shows a pattern of
failures due to timeouts, with a different test being
hit/stopped (and the following tests planned in the
.js file not being run) each time, which can be used
as evidence of how the errors look like.
[Regression Potential]
* Low. The changes only affect build-time execution,
not anything that has runtime changes.
* The differences in build dependencies between the
original build and a rebuild with this patch should
not be carried in the package, as nodejs modules
should mostly be binary-indepedent/arch-all, thus not make
it into the package but rather are sourced at runtime.
Bug Description:
The node-tap package in Bionic usually fails autopkgtests
with several intermittent errors caused by timeouts, that
depend directly on the load of the autopkgtest server.
It's very difficult to reproduce these failures on a local
computer that is lightly-loaded, but it's possible to force
them to happen very similarly to the autupkgtest error logs
by manually decreasing/specifying the timeout values.
The same package in Cosmic passes all autopkgtests, and the
code differences between both releases include 2 commits in
Debian that cause the timeout values to be increased.
So, pull those 2 commits from Debian & Cosmic to Bionic, and
allow the tests to finish properly, so to have more reliable
autopkgtest results.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/node-tap/+bug/1793612/+subscriptions
More information about the Ubuntu-sponsors
mailing list