KDE autopkgtest concerns

[Iain Lane]

I'm worried that KDE autopkgtests are a serious strain on the
infrastructure.

  (1) They are C++ applications that rebuild the whole package to run
      their tests. As well as being *very* time and compute intensive,
      this means they test the just-built thing¹ and not the binaries in
      the archive, so they are not testing the things that users are
      getting - which is what autopkgtests are supposed to test.
  (2) When there is a new upstream version uploaded, all the tests run,
      fail, and get retried with all-proposed=1. That is very wasteful.
      This will probably be due to some not-tight-enough dependencies
      but I'm not aware that anybody has worked to fix this.
  (3) Uploads trigger a whole web of rdependency tests, each of which
      have the above problems.

(3) would not be a problem if (1) and (2) were solved, but in the
absence of that it multiplies the problem.

Of course, when the tests are running, they take up a worker instance
which can't be used for something else. The rebuilds also cause 'noisy
neighbour' effects which impact unrelated tests that happen to be
running simultaneously.

Taken together, these are a big cause of slower queue throughput that is
especially bad when (and contribute to) the queues are large.

I've tried to raise this before, many times, but we've never got any
traction. If we can't get some this time, I'm proposing that we
blacklist these tests since they are too much of a strain on our
resources for not very much gain. We are a testing service, not a
continuous rebuild service.

Cheers,

-- 
Iain Lane                                  [ iain at orangesquash.org.uk ]
Debian Developer                                   [ laney at debian.org ]
Ubuntu Developer                                   [ laney at ubuntu.com ]

¹ for example:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-disco/disco/amd64/k/kio/20190116_153902_bc22d@/log.gz:
  /usr/bin/ctest --force-new-ctest-process -j1
  Test project /tmp/autopkgtest.PRFpOs/build.vhn/src/obj-x86_64-linux-gnu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20190116/417b73f2/attachment.sig>