[Bug 1888575] Re: Split motd-news config into a new package

[Andreas Hasenack]

** Description changed:

  [Impact]
  The motd-news script is largely useless for desktop users, as they rarely login via a text console. It makes more sense for server users.
  
  We can use package dependencies to have the motd-news script enabled on servers, but disabled on desktops, and still handle upgrades. This is the plan:
  - move /etc/default/motd-news from base-files into a new binary package (motd-news-config, produced by src:base-files)
  - have ubuntu-server depend on motd-news-config
  - have base-files break current ubuntu-server, so that if base-files if upgraded and ubuntu-server is installed, ubuntu-server will also be upgraded to the new version which has the depends on motd-news-config
  
  Care must be taken to preserve a changed /etc/default/motd-news when the
  upgrade installs the new motd-news-config package. For example, on a
  server that has set ENABLED=0 in /etc/default/motd-news and upgrades to
  the new base-files and ubuntu-server, and gets the new motd-config-news
  package, ENABLED=0 must remain set.
  
  [Test Case]
  a) base-files installed, ubuntu-server installed, unmodified /e/d/motd-news
  apt install base-files
  - upgrades ubuntu-server
  - installs motd-news-config
  - /e/d/motd-news remains, motd-news remains enabled
  
  b) base-files installed, ubuntu-server installed, modified /e/d/motd-news
  apt install base-files
  - upgrades ubuntu-server
  - installs motd-news-config
  - /e/d/motd-news remains with the original modification
  
  c) base-files installed, ubuntu-server not installed, unmodified /e/d/motd-news
  apt install base-files
  - upgrades base-files
  - removes /e/d/motd-news
  - motd-news is disabled
  
  d) base-files installed, ubuntu-server not installed, modified /e/d/motd-news
  apt install base-files
  - upgrades base-files
  - /e/d/motd-news gets renamed to backup
  - motd-news is disabled
  
  e) removing motd-news-config will also remove ubuntu-server (since it's
  a depends, and not a recommends)
  
  f) upgrading just ubuntu-server should pull motd-news-config in, and
  force-upgrade base-files
  
  g) Removing motd-news-server leaves /e/d/motd-news around; purging motd-
  news-server removes the /e/d/motd-news config file
  
  h) base-files installed, ubuntu-server installed, removed /e/d/motd-news
  - apt install base-files
  - upgrades base-files, upgrades ubuntu-server, installs motd-news-config
  - /e/d/motd-news is installed with ENABLED=0
  
  i) base-files installed, ubuntu-server NOT installed, removed e/d/motd-news
  - apt install base-files
  - base-files is upgraded
  - no /e/d/motd-news is installed, motd-news remains disabled
  
  [Regression Potential]
  This update is about config file ownership transfer: /e/d/motd-news belonged to base-files, now it belongs to motd-news-config. We tried to handle two important cases here:
  a) /e/d/motd-news config was changed while it belonged to base-files. For example, an user could have set ENABLED=0. We need to transfer that change to the motd-news-config package when it is installed, otherwise this SRU would jsut re-enabled motd-news. This is handled in d/motd-news-config.postinst's configure case.
  
  b) /e/d/motd-news config file was *removed* while it belonged to base-files. In such a case, a normal upgrade of the package (base-files in this example) would not reinstate the file. Much less this upgrade here, which has an explicit rm_conffile maintscript-helper for it. But the motd-news-config package that could be installed in the transaction would place the default config file back, and the default is ENABLED=1. Thus, a system that had motd-news disabled via removing the config file would now have it re-enabled after the upgrade.
- This was trickier to handle, and we do it in base-files's postinst and motd-news-config's  postinst. The drawback is that in one scenario, where just base-files is upgraded, there will be a /e/d/motd-news.wasremoved leftover empty file (see "other info" below for details).
+ This was trickier to handle, and we do it in base-files's postinst and motd-news-config's  postinst. The drawback is that in one scenario, where just base-files is upgraded and /e/d/motd-news was manually removed by the user, there will be a /e/d/motd-news.wasremoved leftover empty file (see "other info" below for details).
  
  In general, the regression risks here are:
  - have motd-news enabled again on a system where it was previously disabled. We tried to envision two ways it would have been disabled (set ENABLED=0, and remove the config file). There are probably others
  - differences in dpkg and/or debhelper behavior in older ubuntu releases leading to unexpected results (should be covered by the test cases from this SRU)
  - xenial in particular is trickier, because src:base-files there does NOT use debhelper, so many of the things we take for granted have to be done by hand
  - have some sort of dpkg postinst or dependency error because of unpredicted scenarios. Certain assumptions are being made, like the renames that dpkg-maintscript-helper does, and that the filename /etc/default/motd-news.wasremoved that I'm touching and verifying is really mine and not something that was there already.
  - the versions I'm breaking/replacing on, and using rm_conffiles on, must be exact. These are the versions today in the archive (2020-08-12):
  
  base-files:
  x: 9.4ubuntu4.12
  b: 10.1ubuntu2.9
  f: 11ubuntu5.1
  g: 11ubuntu10
  
  ubuntu-meta:
  x: 1.361.4
  b: 1.417.4
  f: 1.450.1
  g: 1.452
  
  Which reflect in these relationships in the updated packages:
  Groovy:
  ubuntu-server 1.453:
-   Depends: motd-news-config
+   Depends: motd-news-config
  base-files 11ubuntu11:
-   Breaks: ubuntu-server (<< 1.453)
-   rm_conffile /etc/default/motd-news 11ubuntu11~ base-files
+   Breaks: ubuntu-server (<< 1.453)
+   rm_conffile /etc/default/motd-news 11ubuntu11~ base-files
  motd-news-config 11ubuntu11:
-   Breaks/Replaces: base-files (<< 11ubuntu11)
+   Breaks/Replaces: base-files (<< 11ubuntu11)
  
  Focal:
  ubuntu-server 1.450.2:
-   Depends: motd-news-config
+   Depends: motd-news-config
  base-files 11ubuntu5.2:
-   Breaks: ubuntu-server (<< 1.450.2)
-   rm_conffile /etc/default/motd-news 11ubuntu5.2~ base-files
+   Breaks: ubuntu-server (<< 1.450.2)
+   rm_conffile /etc/default/motd-news 11ubuntu5.2~ base-files
  motd-news-config 11ubuntu5.2:
-   Breaks/Replaces: base-files (<< 11ubuntu5.2)
+   Breaks/Replaces: base-files (<< 11ubuntu5.2)
  
  Bionic:
  ubuntu-server 1.417.5:
-   Depends: motd-news-config
+   Depends: motd-news-config
  base-files 10.1ubuntu2.10:
-   Breaks: ubuntu-server (<< 1.417.5)
-   rm_conffile /etc/default/motd-news 10.1ubuntu2.10~ base-files
+   Breaks: ubuntu-server (<< 1.417.5)
+   rm_conffile /etc/default/motd-news 10.1ubuntu2.10~ base-files
  motd-news-config 10.1ubuntu2.10:
-   Breaks/Replaces: base-files (<< 10.1ubuntu2.10)
+   Breaks/Replaces: base-files (<< 10.1ubuntu2.10)
  
  Xenial:
  ubuntu-server 1.361.5:
-   Depends: motd-news-config
+   Depends: motd-news-config
  base-files 9.4ubuntu4.13:
-   Breaks: ubuntu-server (<< 1.361.5)
-   rm_conffile /etc/default/motd-news 9.4ubuntu4.13~ base-files
+   Breaks: ubuntu-server (<< 1.361.5)
+   rm_conffile /etc/default/motd-news 9.4ubuntu4.13~ base-files
  motd-news-config 9.4ubuntu4.13:
-   Breaks/Replaces: base-files (<< 9.4ubuntu4.13)
+   Breaks/Replaces: base-files (<< 9.4ubuntu4.13)
  
  [Other Info]
  a) Testcase (i) will leave around an empty /etc/default/motd-news.wasremoved file, created by the base-files postinst. This file is removed by the motd-news-config postinst, but since that package doesn't get installed in that particular scenario, the file remains. I toyed with the idea of adding an extra check to base-file's postinst, like this:
  --- a/debian/postinst.in
  +++ b/debian/postinst.in
  @@ -133,7 +133,11 @@ motd_news_config="/etc/default/motd-news"
   if [ ! -e ${motd_news_config} ]; then
     if [ ! -e ${motd_news_config}.dpkg-remove ]; then
       if [ ! -e ${motd_news_config}.dpkg-backup ]; then
  -      touch ${motd_news_config}.wasremoved
  +      # The .wasremoved file only matters if ubuntu-server is installed,
  +      # because that's what will pull in motd-news-config
  +      if dpkg -l ubuntu-server 2>/dev/null | grep -q ^i; then
  +        touch ${motd_news_config}.wasremoved
  +      fi
       fi
     fi
   fi
  
  But deemed it too risky, and not worth further potential regressions.
  
  b) Currently the xenial cloud images, with the exception of the AWS one,
  do not have ubuntu-server installed. This means that this SRU will
  disable motd-news on them, unless ubuntu-server was manually installed
  for some reason. This includes LXD xenial images as well.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to base-files in Ubuntu.
https://bugs.launchpad.net/bugs/1888575

Title:
  Split motd-news config into a new package

Status in base-files package in Ubuntu:
  In Progress
Status in ubuntu-meta package in Ubuntu:
  In Progress
Status in base-files source package in Xenial:
  In Progress
Status in ubuntu-meta source package in Xenial:
  In Progress
Status in base-files source package in Bionic:
  In Progress
Status in ubuntu-meta source package in Bionic:
  In Progress
Status in base-files source package in Focal:
  In Progress
Status in ubuntu-meta source package in Focal:
  In Progress
Status in base-files source package in Groovy:
  In Progress
Status in ubuntu-meta source package in Groovy:
  In Progress

Bug description:
  [Impact]
  The motd-news script is largely useless for desktop users, as they rarely login via a text console. It makes more sense for server users.

  We can use package dependencies to have the motd-news script enabled on servers, but disabled on desktops, and still handle upgrades. This is the plan:
  - move /etc/default/motd-news from base-files into a new binary package (motd-news-config, produced by src:base-files)
  - have ubuntu-server depend on motd-news-config
  - have base-files break current ubuntu-server, so that if base-files if upgraded and ubuntu-server is installed, ubuntu-server will also be upgraded to the new version which has the depends on motd-news-config

  Care must be taken to preserve a changed /etc/default/motd-news when
  the upgrade installs the new motd-news-config package. For example, on
  a server that has set ENABLED=0 in /etc/default/motd-news and upgrades
  to the new base-files and ubuntu-server, and gets the new motd-config-
  news package, ENABLED=0 must remain set.

  [Test Case]
  a) base-files installed, ubuntu-server installed, unmodified /e/d/motd-news
  apt install base-files
  - upgrades ubuntu-server
  - installs motd-news-config
  - /e/d/motd-news remains, motd-news remains enabled

  b) base-files installed, ubuntu-server installed, modified /e/d/motd-news
  apt install base-files
  - upgrades ubuntu-server
  - installs motd-news-config
  - /e/d/motd-news remains with the original modification

  c) base-files installed, ubuntu-server not installed, unmodified /e/d/motd-news
  apt install base-files
  - upgrades base-files
  - removes /e/d/motd-news
  - motd-news is disabled

  d) base-files installed, ubuntu-server not installed, modified /e/d/motd-news
  apt install base-files
  - upgrades base-files
  - /e/d/motd-news gets renamed to backup
  - motd-news is disabled

  e) removing motd-news-config will also remove ubuntu-server (since
  it's a depends, and not a recommends)

  f) upgrading just ubuntu-server should pull motd-news-config in, and
  force-upgrade base-files

  g) Removing motd-news-server leaves /e/d/motd-news around; purging
  motd-news-server removes the /e/d/motd-news config file

  h) base-files installed, ubuntu-server installed, removed /e/d/motd-news
  - apt install base-files
  - upgrades base-files, upgrades ubuntu-server, installs motd-news-config
  - /e/d/motd-news is installed with ENABLED=0

  i) base-files installed, ubuntu-server NOT installed, removed e/d/motd-news
  - apt install base-files
  - base-files is upgraded
  - no /e/d/motd-news is installed, motd-news remains disabled

  [Regression Potential]
  This update is about config file ownership transfer: /e/d/motd-news belonged to base-files, now it belongs to motd-news-config. We tried to handle two important cases here:
  a) /e/d/motd-news config was changed while it belonged to base-files. For example, an user could have set ENABLED=0. We need to transfer that change to the motd-news-config package when it is installed, otherwise this SRU would jsut re-enabled motd-news. This is handled in d/motd-news-config.postinst's configure case.

  b) /e/d/motd-news config file was *removed* while it belonged to base-files. In such a case, a normal upgrade of the package (base-files in this example) would not reinstate the file. Much less this upgrade here, which has an explicit rm_conffile maintscript-helper for it. But the motd-news-config package that could be installed in the transaction would place the default config file back, and the default is ENABLED=1. Thus, a system that had motd-news disabled via removing the config file would now have it re-enabled after the upgrade.
  This was trickier to handle, and we do it in base-files's postinst and motd-news-config's  postinst. The drawback is that in one scenario, where just base-files is upgraded and /e/d/motd-news was manually removed by the user, there will be a /e/d/motd-news.wasremoved leftover empty file (see "other info" below for details).

  In general, the regression risks here are:
  - have motd-news enabled again on a system where it was previously disabled. We tried to envision two ways it would have been disabled (set ENABLED=0, and remove the config file). There are probably others
  - differences in dpkg and/or debhelper behavior in older ubuntu releases leading to unexpected results (should be covered by the test cases from this SRU)
  - xenial in particular is trickier, because src:base-files there does NOT use debhelper, so many of the things we take for granted have to be done by hand
  - have some sort of dpkg postinst or dependency error because of unpredicted scenarios. Certain assumptions are being made, like the renames that dpkg-maintscript-helper does, and that the filename /etc/default/motd-news.wasremoved that I'm touching and verifying is really mine and not something that was there already.
  - the versions I'm breaking/replacing on, and using rm_conffiles on, must be exact. These are the versions today in the archive (2020-08-12):

  base-files:
  x: 9.4ubuntu4.12
  b: 10.1ubuntu2.9
  f: 11ubuntu5.1
  g: 11ubuntu10

  ubuntu-meta:
  x: 1.361.4
  b: 1.417.4
  f: 1.450.1
  g: 1.452

  Which reflect in these relationships in the updated packages:
  Groovy:
  ubuntu-server 1.453:
    Depends: motd-news-config
  base-files 11ubuntu11:
    Breaks: ubuntu-server (<< 1.453)
    rm_conffile /etc/default/motd-news 11ubuntu11~ base-files
  motd-news-config 11ubuntu11:
    Breaks/Replaces: base-files (<< 11ubuntu11)

  Focal:
  ubuntu-server 1.450.2:
    Depends: motd-news-config
  base-files 11ubuntu5.2:
    Breaks: ubuntu-server (<< 1.450.2)
    rm_conffile /etc/default/motd-news 11ubuntu5.2~ base-files
  motd-news-config 11ubuntu5.2:
    Breaks/Replaces: base-files (<< 11ubuntu5.2)

  Bionic:
  ubuntu-server 1.417.5:
    Depends: motd-news-config
  base-files 10.1ubuntu2.10:
    Breaks: ubuntu-server (<< 1.417.5)
    rm_conffile /etc/default/motd-news 10.1ubuntu2.10~ base-files
  motd-news-config 10.1ubuntu2.10:
    Breaks/Replaces: base-files (<< 10.1ubuntu2.10)

  Xenial:
  ubuntu-server 1.361.5:
    Depends: motd-news-config
  base-files 9.4ubuntu4.13:
    Breaks: ubuntu-server (<< 1.361.5)
    rm_conffile /etc/default/motd-news 9.4ubuntu4.13~ base-files
  motd-news-config 9.4ubuntu4.13:
    Breaks/Replaces: base-files (<< 9.4ubuntu4.13)

  [Other Info]
  a) Testcase (i) will leave around an empty /etc/default/motd-news.wasremoved file, created by the base-files postinst. This file is removed by the motd-news-config postinst, but since that package doesn't get installed in that particular scenario, the file remains. I toyed with the idea of adding an extra check to base-file's postinst, like this:
  --- a/debian/postinst.in
  +++ b/debian/postinst.in
  @@ -133,7 +133,11 @@ motd_news_config="/etc/default/motd-news"
   if [ ! -e ${motd_news_config} ]; then
     if [ ! -e ${motd_news_config}.dpkg-remove ]; then
       if [ ! -e ${motd_news_config}.dpkg-backup ]; then
  -      touch ${motd_news_config}.wasremoved
  +      # The .wasremoved file only matters if ubuntu-server is installed,
  +      # because that's what will pull in motd-news-config
  +      if dpkg -l ubuntu-server 2>/dev/null | grep -q ^i; then
  +        touch ${motd_news_config}.wasremoved
  +      fi
       fi
     fi
   fi

  But deemed it too risky, and not worth further potential regressions.

  b) Currently the xenial cloud images, with the exception of the AWS
  one, do not have ubuntu-server installed. This means that this SRU
  will disable motd-news on them, unless ubuntu-server was manually
  installed for some reason. This includes LXD xenial images as well.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/base-files/+bug/1888575/+subscriptions