[Bug 1882069] Re: DistUpgradeController.py key 'devRelease' not set correctly
Marcel Sachtleben
1882069 at bugs.launchpad.net
Thu Jun 4 11:15:03 UTC 2020
Here is the content of /var/log/dist-upgrade/main.log (file path may
differ):
root at snoobeam:/# cat /var/log/dist-upgrade/main.log
2020-05-28 21:03:34,995 INFO Using config files '['./DistUpgrade.cfg']'
2020-05-28 21:03:34,995 INFO uname information: 'Linux snoobeam.com 5.4.0-33-generic #37-Ubuntu SMP Thu May 21 12:53:59 UTC 2020 x86_64'
2020-05-28 21:03:35,113 INFO apt version: '1.9.4ubuntu0.1'
2020-05-28 21:03:35,113 INFO python version: '3.7.5 (default, Apr 19 2020, 20:18:17)
[GCC 9.2.1 20191008]'
2020-05-28 21:03:35,114 INFO release-upgrader version '20.04.19' started
2020-05-28 21:03:35,129 INFO locale: 'en_US' 'UTF-8'
2020-05-28 21:03:35,202 INFO screen could not be run
2020-05-28 21:03:35,271 DEBUG Using 'DistUpgradeViewText' view
2020-05-28 21:03:35,337 DEBUG enable dpkg --force-overwrite
2020-05-28 21:03:35,436 DEBUG creating statefile: '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'
2020-05-28 21:03:36,399 DEBUG lsb-release: 'eoan'
2020-05-28 21:03:36,400 DEBUG _pythonSymlinkCheck run
2020-05-28 21:03:36,402 DEBUG openCache()
2020-05-28 21:03:36,403 DEBUG quirks: running PreCacheOpen
2020-05-28 21:03:36,403 DEBUG running Quirks.PreCacheOpen
2020-05-28 21:03:36,671 DEBUG /openCache(), new cache size 10843
2020-05-28 21:03:36,672 DEBUG need_server_mode(): can not find a desktop meta package or key deps, running in server mode
2020-05-28 21:03:36,672 DEBUG checkViewDepends()
2020-05-28 21:03:36,678 DEBUG running doUpdate() (showErrors=False)
2020-05-28 21:03:37,783 DEBUG openCache()
2020-05-28 21:03:38,020 DEBUG /openCache(), new cache size 10843
2020-05-28 21:03:38,020 DEBUG doPostInitialUpdate
2020-05-28 21:03:38,020 DEBUG quirks: running focalPostInitialUpdate
2020-05-28 21:03:38,021 DEBUG running Quirks.focalPostInitialUpdate
2020-05-28 21:03:38,301 DEBUG MetaPkgs:
2020-05-28 21:03:38,692 DEBUG no PkgRecord found for 'fonts-urw-base35', skipping
2020-05-28 21:03:38,998 DEBUG no PkgRecord found for 'mariadb-client-10.1', skipping
2020-05-28 21:03:38,998 DEBUG no PkgRecord found for 'mariadb-server-10.1', skipping
2020-05-28 21:03:39,016 DEBUG no PkgRecord found for 'pgadmin4-apache2', skipping
2020-05-28 21:03:39,016 DEBUG no PkgRecord found for 'pgdg-keyring', skipping
2020-05-28 21:03:39,022 DEBUG no PkgRecord found for 'postgresql-12', skipping
2020-05-28 21:03:39,102 DEBUG Foreign:
2020-05-28 21:03:39,102 DEBUG Obsolete: apt-transport-https aptitude aptitude-common binfmt-support ca-certificates-mono certbot cli-common command-not-found-data cpp-7 cpp-8 daemon denyhosts dovecot-ldap dovecot-lmtpd dovecot-managesieved dovecot-mysql dovecot-sieve fail2ban fonts-dejavu fonts-glyphicons-halflings g++-7 galera-3 gcc-10-base gcc-7 gcc-7-base gcc-8 gcc-8-base gcc-8-multilib gir1.2-spiceclientglib-2.0 inotify-tools jenkins ldap-account-manager ldap-auth-client ldap-auth-config lib32gcc-8-dev lib32mpx2 libaopalliance-java libapache-pom-java libapache2-mod-php7.2 libapache2-mod-rpaf libapt-inst2.0 libapt-pkg5.0 libasan4 libatinject-jsr330-api-java libcdi-api-java libcilkrts5 libcommons-cli-java libcommons-io-java libcommons-lang3-java libcommons-parent-java libcrypto++-dev libcrypto++6 libcwidget3v5 libdbd-mysql-perl libdbus-glib2.0-cil libdbus-glib2.0-cil-dev libdbus2.0-cil libdbus2.0-cil-dev libdouble-conversion3 libffi7 libfstrm0 libgcc-7-dev libgcc-8-dev libgeronimo-annotation-1.3-spec-java libgeronimo-interceptor-3.0-spec-java libglib2.0-cil libglib2.0-cil-dev libglib2.0-tests libguava-java libguice-java libhawtjni-runtime-java libhiredis0.14 libicu60 libinotifytools0 libip4tc0 libip6tc0 libjansi-java libjansi-native-java libjemalloc2 libjs-bootstrap libjs-requirejs libjson-c3 libjsr305-java liblua5.1-0 liblua5.1-0-dev libluajit-5.1-2 libluajit-5.1-common libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java libmaven3-core-java libmono-btls-interface4.0-cil libmono-corlib4.5-cil libmono-i18n-west4.0-cil libmono-i18n4.0-cil libmono-posix4.0-cil libmono-security4.0-cil libmono-system-configuration4.0-cil libmono-system-core4.0-cil libmono-system-numerics4.0-cil libmono-system-security4.0-cil libmono-system-xml4.0-cil libmono-system4.0-cil libmpx2 libnode64 libnss-ldap libpam-ldap libparse-debianchangelog-perl libphodav-2.0-0 libphodav-2.0-common libplexus-cipher-java libplexus-classworlds-java libplexus-component-annotations-java libplexus-interpolation-java libplexus-sec-dispatcher-java libplexus-utils2-java libprotobuf-c1 libpython-all-dev libpython-dev libpython-stdlib libpython2-dev libpython2-stdlib libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib libqt5core5a libqt5glib-2.0-0 libsisu-inject-java libsisu-plexus-java libslf4j-java libspice-client-glib-2.0-8 libspice-client-glib-2.0-dev libstdc++-7-dev libubsan0 libwagon-file-java libwagon-http-shaded-java libwagon-provider-api-java libx32gcc-8-dev libxapian30 libzip5 linux-headers-5.4.0-33 linux-headers-5.4.0-33-generic linux-image-5.4.0-33-generic linux-modules-5.4.0-33-generic linux-modules-extra-5.4.0-33-generic lua-bitop lua-cjson lua5.1 luajit mariadb-client-10.3 mariadb-client-core-10.3 mariadb-common mariadb-server mariadb-server-10.3 mariadb-server-core-10.3 maven mono-4.0-gac mono-gac mono-runtime mono-runtime-common mono-runtime-sgen multiarch-support nodejs nodejs-doc nscd nslcd nslcd-utils php-fpm php-imagick php-monolog php-phpseclib php-psr-log php-zip php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-fpm php7.2-gd php7.2-json php7.2-ldap php7.2-opcache php7.2-readline php7.2-xml php7.2-zip php7.3-fpm php7.3-zip php7.4 php7.4-cli php7.4-common php7.4-curl php7.4-fpm php7.4-gd php7.4-json php7.4-ldap php7.4-opcache php7.4-readline php7.4-xml php7.4-zip phpldapadmin postgresql-10 postgresql-client-10 python python-all python-all-dev python-asn1crypto python-certbot-nginx python-cffi-backend python-configparser python-crypto python-cryptography python-dbus python-dev python-entrypoints python-enum34 python-gi python-ipaddress python-keyring python-keyrings.alt python-minimal python-pip python-pip-whl python-pkg-resources python-pyicu python-secretstorage python-setuptools python-six python-wheel python-xdg python2 python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal python3-acme python3-certbot python3-certbot-nginx python3-configargparse python3-josepy python3-keyrings.alt python3-mock python3-parsedatetime python3-pip python3-requests-toolbelt python3-venv python3-virtualenv python3-wheel python3-zope.component python3-zope.event python3-zope.hookable python3.7-venv qttranslations5-l10n ragel redis-server redis-tools resolvconf rspamd spice-client-glib-usb-acl-helper ttf-dejavu-core unbound unbound-anchor virtualenv
2020-05-28 21:03:39,103 DEBUG updateSourcesList()
2020-05-28 21:03:39,123 DEBUG rewriteSourcesList() with mirror_check
2020-05-28 21:03:39,124 DEBUG ['ubuntu-minimal', 'ubuntu-standard']
2020-05-28 21:03:39,124 DEBUG Checking pkg: ubuntu-minimal
2020-05-28 21:03:39,125 DEBUG Checking pkg: ubuntu-standard
2020-05-28 21:03:39,126 DEBUG examining: 'deb http://archive.ubuntu.com/ubuntu eoan main restricted'
2020-05-28 21:03:39,126 DEBUG verifySourcesListEntry: deb http://archive.ubuntu.com/ubuntu focal main restricted
2020-05-28 21:03:39,126 DEBUG url_downloadable: http://archive.ubuntu.com/ubuntu/dists/focal/Release
2020-05-28 21:03:39,126 DEBUG s='http' n='archive.ubuntu.com' p='/ubuntu/dists/focal/Release' q='' f=''
2020-05-28 21:03:39,176 DEBUG entry 'deb http://archive.ubuntu.com/ubuntu focal main restricted' updated to new dist
2020-05-28 21:03:39,176 DEBUG examining: 'deb-src http://archive.ubuntu.com/ubuntu eoan main restricted'
2020-05-28 21:03:39,176 DEBUG entry 'deb-src http://archive.ubuntu.com/ubuntu focal main restricted' updated to new dist
2020-05-28 21:03:39,177 DEBUG examining: 'deb http://archive.ubuntu.com/ubuntu eoan-updates main restricted'
2020-05-28 21:03:39,177 DEBUG entry 'deb http://archive.ubuntu.com/ubuntu focal-updates main restricted' updated to new dist
2020-05-28 21:03:39,177 DEBUG examining: 'deb-src http://archive.ubuntu.com/ubuntu eoan-updates main restricted'
2020-05-28 21:03:39,177 DEBUG entry 'deb-src http://archive.ubuntu.com/ubuntu focal-updates main restricted' updated to new dist
2020-05-28 21:03:39,182 DEBUG running doUpdate() (showErrors=True)
2020-05-28 21:03:41,646 DEBUG openCache()
2020-05-28 21:03:42,116 DEBUG /openCache(), new cache size 8801
2020-05-28 21:03:42,117 DEBUG need_server_mode(): can not find a desktop meta package or key deps, running in server mode
2020-05-28 21:03:42,936 ERROR not handled exception:
Traceback (most recent call last):
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeCache.py", line 618, in distUpgrade
self.upgrade(True)
File "/usr/lib/python3/dist-packages/apt/cache.py", line 388, in upgrade
self._depcache.upgrade(dist_upgrade)
apt_pkg.Error: E:Error, pkgProblemResolver::Resolve generated breaks,
this may be caused by held packages.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/configparser.py", line 788, in get
value = d[option]
File "/usr/lib/python3.7/collections/__init__.py", line 916, in __getitem__
return self.__missing__(key) # support subclasses that define __missing__
File "/usr/lib/python3.7/collections/__init__.py", line 908, in __missing__
raise KeyError(key)
KeyError: 'devrelease'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/focal", line 8, in <module>
sys.exit(main())
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeMain.py", line 238, in main
if app.run():
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeController.py", line 2082, in run
return self.fullUpgrade()
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeController.py", line 1991, in fullUpgrade
if not self.askDistUpgrade():
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeController.py", line 1152, in askDistUpgrade
changes = self.calcDistUpgrade()
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeController.py", line 1115, in calcDistUpgrade
if not self.cache.distUpgrade(self._view, self.serverMode, self._partialUpgrade):
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeCache.py", line 241, in wrapper
res = f(*args, **kwargs)
File "/tmp/ubuntu-release-upgrader-qzfkvt6y/DistUpgrade/DistUpgradeCache.py", line 668, in distUpgrade
self.config.get("Options", "devRelease") == "True":
File "/usr/lib/python3.7/configparser.py", line 791, in get
raise NoOptionError(option, section)
configparser.NoOptionError: No option 'devrelease' in section: 'Options'
2020-05-28 21:03:42,936 DEBUG running apport_crash()
2020-05-28 21:03:42,936 ERROR failed to import apport python module, can't generate crash: No module named 'apport_python_hook'
2020-05-28 21:03:42,936 DEBUG can't find apport
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1882069
Title:
DistUpgradeController.py key 'devRelease' not set correctly
Status in ubuntu-release-upgrader package in Ubuntu:
New
Bug description:
I encountered this bug when execution do-release-upgrade on an Ubuntu 19.10 machine.
The release upgrade crashed ungracefully not calling abort() which should rollback all changes up to that point.
So I ended up with kind of a broken packages and configuration hell on my server machine.
I finally was able to perform the release upgrade by using apt / dpkg manually and resolving package dependencies and conflicts by hand.
The bug is on actually two places:
DistUpgradeController.py (parameter devRelease not set at all due to
wrong indendation of else block)
DistUpgradeCache.py (using unsafe code within except block: accessing
undefined key 'devRelease' and provoke KeyError)
In general: the idea of except or catch or whatever is not just
execute different code logic in case of exception. It is about doing
the most basic stuff in case anything goes wrong (which should be as
exception (error) safe as possible)
Bad practice:
try
doSomething();
catch exception
ignoreException();
doSomethingElse();
Good practice:
try
doSomethingUnsafe();
catch exception e
logAndHandleException(e);
useAnotherTryIfAdditionalLogicIsRequired();
That kind of bad structure costed me 1 day of error analysis (I am not
a python guy) and another 1 day to revert the things from the failed
upgrade.
The bugs:
Since I can not find any source code git repository, the next lines target the python module which can be found in /usr/lib/python3/dist-packages/DistUpgrade (Ubuntu 19.10 / Ubuntu 20.04 LTS)
(line numbers may differ)
Major Bug:
DistUpgradeController.py:138-139 (wrong indentation of else block)
Current:
if self.options:
if self.options.devel_release:
self.config.set("Options","devRelease", "True")
else:
self.config.set("Options","devRelease", "False")
Should be:
if self.options:
if self.options.devel_release:
self.config.set("Options","devRelease", "True")
else:
self.config.set("Options","devRelease", "False")
Minor Bug:
DistUpgradeCache.py:651-694 (except block)
Error arised in lines 667-668
elif self.config.get("Options", "foreignPkgs") == "False" and \
self.config.get("Options", "devRelease") == "True":
cause "devRelease" was not set on "Options" (see Major bug)
except block should not excecute unsafe code cause its job is error handling and roll back.
please make the entire block more fail safe (also use additional "try" if neccessary)
Since it seems like that accessing keys in python is not safe (reminds me of javascript)
it should be always checked if the key does even exist before accessing it.
Thanks for taking care of that.
Edit: I found the attachment feature after writing this report so
please find the DistUpgradeController.py attached containing the major
bug.
Please keep me up to date
marcel.sachtleben at snoobeam.com
Best Regards
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/1882069/+subscriptions
More information about the foundations-bugs
mailing list