[Bug 1771102] Re: 17.10->18.04 LTS hangs, CPU stalls in apt-pkg cache

[Alexander Sack]

Brian, I'm sorry but I had to upgrade to 18.04 LTS by reinstalling the
whole system. No idea why apt-clone would hang like that but it seems to
definitely be a bug.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1771102

Title:
  17.10->18.04 LTS hangs, CPU stalls in apt-pkg cache

Status in python-apt package in Ubuntu:
  New
Status in ubuntu-release-upgrader package in Ubuntu:
  New

Bug description:
  When I do as root:

  do-release-upgrade

  The bionic tarball is downloaded and the python based "bionic" process
  is forked. On the screen I see the first "Reading cache..." message
  and then the process spikes the CPU to 100% where it stays there
  forever (I waited 15-20 minutes and there was no change in CPU state).

  Here is the dist-upgrade log file:
  2018-05-14 07:17:19,308 INFO uname information: 'Linux kitt 4.13.0-41-generic #46-Ubuntu SMP Wed May 2 13:38:30 UTC 2018 x86_64'
  2018-05-14 07:17:19,594 INFO apt version: '1.5.1'
  2018-05-14 07:17:19,594 INFO python version: '3.6.3 (default, Oct  3 2017, 21:45:48)
  [GCC 7.2.0]'
  2018-05-14 07:17:19,596 INFO release-upgrader version '18.04.17' started
  2018-05-14 07:17:19,600 INFO locale: 'en_US' 'UTF-8'
  2018-05-14 07:17:19,605 INFO screen could not be run
  2018-05-14 07:17:19,744 DEBUG Using 'DistUpgradeViewText' view
  2018-05-14 07:17:19,788 DEBUG enable dpkg --force-overwrite
  2018-05-14 07:17:19,817 DEBUG creating statefile: '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'

  strace output of the "bionic" process:

  $ sudo strace -f -p 6469
  strace: Process 6469 attached with 3 threads
  [pid  6546] restart_syscall(<... resuming interrupted poll ...> <unfinished ...>
  [pid  6547] restart_syscall(<... resuming interrupted poll ...>strace: [ Process PID=6469 runs in x32 mode. ]

  top output:
     PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
    6469 root      20   0  270048  33108  15728 R  99.7  0.1   0:38.85 bionic
    6555 pisymbol  20   0   46736   4376   3344 R   0.7  0.0   0:00.04 top

  Looking at this further, it seems that the installer sets the the
  "rootdir=/" when it initiates the python-apt's Cache class. This
  hangs. I can reproduce this by simply changing it's test code to pass
  rootdir="/" and it hangs. It sorta smells like a deadlock issue though
  deep in the C++ module (apt_pkg) that backs the python one (GetLock?).
  But I'm not 100% sure.

  This behavior is seen on my 32-core Threadripper machine (AMD64).

  This is 17.10 latest with:

  $ dpkg -l python-apt libapt-pkg5.0
  Desired=Unknown/Install/Remove/Purge/Hold
  | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
  |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
  ||/ Name                                           Version                      Architecture                 Description
  +++-==============================================-============================-============================-=================================================================================================
  ii  libapt-pkg5.0:amd64                            1.5.1                        amd64                        package management runtime library
  ii  python-apt                                     1.4.0~beta3build2            amd64                        Python interface to libapt-pkg

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/1771102/+subscriptions