[Bug 1793612] Re: node-tap in bionic fails autopkgtests due to timeouts

[Mauricio Faria de Oliveira]

** Description changed:

+ SRU Template:
+ 
+ [Impact]
+ 
+  * node-tap autopkgtests intermittently fail due to timeouts
+    depending on the load in the server running the tests.
+ 
+  * This does not provide reliable test results, misleading
+    the analysis of new code changes to this package, or
+    other packages which this package depends on (as that
+    triggers a re-test for this package).
+ 
+  * This requires manual effort to re-trigger tests just to
+    verify whether the errors are reproducible/intermittent.
+ 
+ [Test Case]
+ 
+  * It's possible to reproduce the failures due to short
+    timeout values by manually specifying shorter timeouts
+    than the defaults, either via TAP_TIMEOUT environment
+    variable or '-t TIMEOUT' parameter, and in setTimeout()
+    calls (e.g., in test/spawn.js);
+    Then run the test suite in either one of these methods:
+ 
+  * Run tests like 'debian/rules override_dh_auto_test'.
+ 
+  * Run autopkgtest locally.
+ 
+  * The autopkgtest server log history shows a pattern of
+    failures due to timeouts, with a different test being
+    hit/stopped (and the following tests planned in the
+    .js file not being run) each time, which can be used
+    as evidence of how the errors look like.
+ 
+ [Regression Potential]
+ 
+  * Low. The changes only affect build-time execution,
+    not anything that has runtime changes.
+ 
+  * The differences in build dependencies between the
+    original build and a rebuild with this patch should
+    not be carried in the package, as nodejs modules
+    should mostly be binary-indepedent/arch-all, thus not make
+    it into the package but rather are sourced at runtime.
+ 
+ Bug Description:
+ 
  The node-tap package in Bionic usually fails autopkgtests
  with several intermittent errors caused by timeouts, that
  depend directly on the load of the autopkgtest server.
  
  It's very difficult to reproduce these failures on a local
  computer that is lightly-loaded, but it's possible to force
  them to happen very similarly to the autupkgtest error logs
  by manually decreasing/specifying the timeout values.
  
  The same package in Cosmic passes all autopkgtests, and the
  code differences between both releases include 2 commits in
  Debian that cause the timeout values to be increased.
  
  So, pull those 2 commits from Debian & Cosmic to Bionic, and
  allow the tests to finish properly, so to have more reliable
  autopkgtest results.

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1793612

Title:
  node-tap in bionic fails autopkgtests due to timeouts

Status in node-tap package in Ubuntu:
  Fix Released
Status in node-tap source package in Bionic:
  New

Bug description:
  SRU Template:

  [Impact]

   * node-tap autopkgtests intermittently fail due to timeouts
     depending on the load in the server running the tests.

   * This does not provide reliable test results, misleading
     the analysis of new code changes to this package, or
     other packages which this package depends on (as that
     triggers a re-test for this package).

   * This requires manual effort to re-trigger tests just to
     verify whether the errors are reproducible/intermittent.

  [Test Case]

   * It's possible to reproduce the failures due to short
     timeout values by manually specifying shorter timeouts
     than the defaults, either via TAP_TIMEOUT environment
     variable or '-t TIMEOUT' parameter, and in setTimeout()
     calls (e.g., in test/spawn.js);
     Then run the test suite in either one of these methods:

   * Run tests like 'debian/rules override_dh_auto_test'.

   * Run autopkgtest locally.

   * The autopkgtest server log history shows a pattern of
     failures due to timeouts, with a different test being
     hit/stopped (and the following tests planned in the
     .js file not being run) each time, which can be used
     as evidence of how the errors look like.

  [Regression Potential]

   * Low. The changes only affect build-time execution,
     not anything that has runtime changes.

   * The differences in build dependencies between the
     original build and a rebuild with this patch should
     not be carried in the package, as nodejs modules
     should mostly be binary-indepedent/arch-all, thus not make
     it into the package but rather are sourced at runtime.

  Bug Description:

  The node-tap package in Bionic usually fails autopkgtests
  with several intermittent errors caused by timeouts, that
  depend directly on the load of the autopkgtest server.

  It's very difficult to reproduce these failures on a local
  computer that is lightly-loaded, but it's possible to force
  them to happen very similarly to the autupkgtest error logs
  by manually decreasing/specifying the timeout values.

  The same package in Cosmic passes all autopkgtests, and the
  code differences between both releases include 2 commits in
  Debian that cause the timeout values to be increased.

  So, pull those 2 commits from Debian & Cosmic to Bionic, and
  allow the tests to finish properly, so to have more reliable
  autopkgtest results.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/node-tap/+bug/1793612/+subscriptions