[Bug 1902830] Re: autopkgtest fails in hirsute - test_clone_upgrade.py

Wed Nov 4 10:49:43 UTC 2020

Actually one can debug a bit via:
$ export APT_CLONE_DEBUG_RESOLVER=1
$ PYTONPATH=.. python3.8 test_clone_upgrade.py

The libsrt1-gnutls that shows up as proper value in the kubuntu-desktop is mentioned then as:
  Installing libsrt1-gnutls:amd64 as Depends of libavformat58:amd64    
But it is listed the same way when handling xubuntu-desktop and missing there.

 libsrt1-gnutls | 1.4.1-5build1 | groovy/universe  | amd64, arm64, armhf, i386, ppc64el, riscv64, s390x
 libsrt1-gnutls | 1.4.2-1       | hirsute/universe | amd64, arm64, armhf, i386, ppc64el, riscv64, s390x

So it listed a package that is upgraded.
When doing the initial grab of installed packages it logged
libsrt1-gnutls 1.4.1-5build1 1

The test does
1. create an env of the -2 release (groovy in this case)
   old = self._create_fake_upgradable_root(supported[-2], meta=meta)
2. stores a package selection list of OLD in tarball
3. create an env of the -1 release (hirsute in this case)
   new = self._create_fake_upgradable_root(supported[-1], meta=meta)
4. creates an apt-cache of the NEW env
   cache = apt.Cache(rootdir=new)
5. restores package selection of the "clone of NEW" to the state of "OLD" from tarball

It expects that there are upgrades available under the meta packages of
ubuntu-standard""ubuntu-desktop" "kubuntu-desktop" and "xubuntu-
desktop". But in the beginning of a release (or at any later time if
there are apt network issues) there are none.

I've found that we can make this much more reliable (especially early on
when -dev ~= -dev-1 by comparing it with the first release that distro
info reports.

With that I have found a fix that eliminates the false positive.
=> https://code.launchpad.net/~paelzer/apt-clone/apt-clone-fix-fails-early-cycle/+merge/393310

This would still be a new upload, so we could accept the badtest now as-
is and upload a fixed version later ... up to you (release team people).

@mvo are you still reviewing/merging/uploading those MPs to apt-clone?

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt-clone in Ubuntu.
https://bugs.launchpad.net/bugs/1902830

Title:
  autopkgtest fails in hirsute - test_clone_upgrade.py

Status in apt-clone package in Ubuntu:
  Triaged

Bug description:
  It seems this autopkgtest is broken in 21.04 and the issue is
  affecting all architecture at the moment.

  There is nothing yet in -proposed that would indicate a fix is on the
  way (if it would be an upload to the package).

  # The Issue

  ======================================================================
  FAIL: test_clone_upgrade_synthetic (__main__.TestCloneUpgrade)
  test clone upgrade with on-the-fly generated chroots
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "test_clone_upgrade.py", line 59, in test_clone_upgrade_synthetic
      self.assertTrue(len(cache.get_changes()) > 0)
  AssertionError: False is not true

  ----------------------------------------------------------------------
  Ran 2 tests in 12.801s

  Fail log:
  https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-hirsute/hirsute/amd64/a/apt-clone/20201027_023127_1a678@/log.gz
  And a bunch of retries on the infra by

  # Retries

  vorlon:
  https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac
  /autopkgtest-hirsute/hirsute/amd64/a/apt-
  clone/20201028_172522_e07e0@/log.gz

  juliank:
  https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac
  /autopkgtest-hirsute/hirsute/amd64/a/apt-
  clone/20201028_095059_2ef15@/log.gz

  bdmurray:
  https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac
  /autopkgtest-hirsute/hirsute/amd64/a/apt-
  clone/20201104_010337_b3a3a@/log.gz

  But I found no bug that would document the insights so far.
  Hence I'm filing this one and set the mentioned people on CC here.
  Especially thins this test blocks python-apt which breaks automated testing using add-apt-repository I'd expect more people will come by and take a look - so documenting a bit can't hurt in this case.

  # Repro

  This fails in local VM autokgtest with/without the packages in
  proposed.

  # History

  Interestingly there seems to be a pattern of this hitting early -dev cycles (in some of them even being an issue up to late in the cycle).
  Groovy:
  https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-groovy/groovy/amd64/a/apt-clone/20200502_094132_932ac@/log.gz
  Focal:
  https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/amd64/a/apt-clone/20191020_095345_3b0ca@/log.gz
  Bionic:
  https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/a/apt-clone/20171026_195549_b2b60@/log.gz

  Therefore this might - for some of us - be a known re-occuring case
  and just wait for something (an image pushed, a release being mentioned
  in a data file, ...).
  The fact that juliank+bdmurray+vorlon have retried the test makes me think
  it is expected to work already but doesn't?

  Looking at test hints we have had:
  badtest (2015)
  https://git.launchpad.net/~ubuntu-release/britney/+git/hints-ubuntu/commit/?id=32c975e7cb91000fde6a1590b2a34b6677f1fb2c
  badtest (2018)
  https://git.launchpad.net/~ubuntu-release/britney/+git/hints-ubuntu/commit/?id=8bdef30c2deb13ff337d27fd2c1fb3efbc8c8d6a
  (mass) convert badtest to a reset (2020):
  https://git.launchpad.net/~ubuntu-release/britney/+git/hints-ubuntu/commit/?id=22d2cf67f10a1a5322670dcc9b40bbfeed250a7a

  That finally got me to bug 1510060.
  Seems this keeps to continue coming and going.

  I'll maybe propose to change it back to a badtest.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt-clone/+bug/1902830/+subscriptions