[Bug 2155240] Re: masakari-hostmonitor fails to start: masakarimonitors.cmd not installed

[Nick Rosbrook]

Hello Guillaume, or anyone else affected,

Accepted masakari-monitors into resolute-proposed. The package will
build now and be available at
https://launchpad.net/ubuntu/+source/masakari-monitors/21.0.0-0ubuntu1.1
in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
resolute to verification-done-resolute. If it does not fix the bug for
you, please add a comment stating that, and change the tag to
verification-failed-resolute. In either case, without details of your
testing we will not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: masakari-monitors (Ubuntu Resolute)
       Status: In Progress => Fix Committed

** Tags added: verification-needed-resolute

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2155240

Title:
  masakari-hostmonitor fails to start: masakarimonitors.cmd not
  installed

Status in Ubuntu Cloud Archive gazpacho series:
  New
Status in masakari-monitors package in Ubuntu:
  Fix Released
Status in masakari-monitors source package in Resolute:
  Fix Committed
Status in masakari-monitors source package in Stonking:
  Fix Released

Bug description:
  [ Impact ]

   * In Resolute, `masakari-monitors` 21.0.0-0ubuntu1 generates the
     monitor console scripts but installs only the top-level
     `masakarimonitors` Python package. Subpackages such as
     `masakarimonitors.cmd`, `masakarimonitors.hostmonitor`,
     `masakarimonitors.instancemonitor`, and related modules are omitted.

   * As a result, `masakari-hostmonitor` fails immediately on startup
  with:

         ModuleNotFoundError: No module named 'masakarimonitors.cmd'

     The same package-discovery defect can affect the other Masakari monitor
     entry points because they also import modules below
     `masakarimonitors.cmd`.

   * This prevents Masakari host monitoring from starting, so deployments that
     rely on Masakari monitors for host failure detection cannot use the
     packaged monitor services reliably.

   * The upload fixes the package metadata by using setuptools package
     discovery for `masakarimonitors*`, so all subpackages needed by the
     generated console scripts are installed. It also corrects the package
     autopkgtest import check so the test imports the real Python module name,
     `masakarimonitors`.

  [ Test Plan ]

   * Use a clean Resolute system or container with the affected package
     available. Enable the normal Ubuntu archive pocket that contains
     `masakari-monitors` before testing the pre-update package.

   * Reproduce the bug with the affected package:

         sudo apt update
         sudo apt install masakari-host-monitor python3-masakari-monitors
         python3 -c 'import masakarimonitors; print(masakarimonitors)'
         python3 -c 'import masakarimonitors.cmd.hostmonitor'
         masakari-hostmonitor --help

     On the affected package, the top-level import may succeed, but importing
     `masakarimonitors.cmd.hostmonitor` or starting `masakari-hostmonitor`
     fails with `ModuleNotFoundError: No module named 'masakarimonitors.cmd'`.

   * Enable `resolute-proposed`, install the SRU candidate, and verify the
     package version is the candidate version:

         sudo apt update
         sudo apt install masakari-host-monitor python3-masakari-monitors
         apt-cache policy masakari-host-monitor python3-masakari-monitors

   * Verify that all monitor console-script target modules import:

         python3 - <<'PY'
         import importlib

         modules = [
             "masakarimonitors.cmd.hostmonitor",
             "masakarimonitors.cmd.instancemonitor",
             "masakarimonitors.cmd.introspectiveinstancemonitor",
             "masakarimonitors.cmd.processmonitor",
         ]

         for module in modules:
             importlib.import_module(module)
             print(f"OK: {module}")
         PY

   * Verify that the generated entry point reaches option parsing instead of
     failing during import:

         masakari-hostmonitor --help >/tmp/masakari-hostmonitor.help
         test -s /tmp/masakari-hostmonitor.help

   * Run the package autopkgtests for the SRU candidate. The important checks
     are the Python import check for `masakarimonitors` and the explicit import
     coverage for the monitor entry-point modules:

         autopkgtest masakari-monitors -- schroot resolute-amd64

     Use the testbed name appropriate for the SRU verifier's
  environment.

  [ Where problems could occur ]

   * The package-discovery change affects which Python modules are installed
     into `python3-masakari-monitors`. If it accidentally includes too little,
     one or more monitor services will continue to fail at import time. If it
     includes unexpected extra files, there could be packaging churn or
     conflicts with files expected to be owned by a different binary package.

   * If the installed subpackages are present but dependency metadata is
     incomplete, failures may move from `ModuleNotFoundError` for
     `masakarimonitors.cmd` to import errors for runtime dependencies used by
     the monitor modules. This would show up when running the import checks or
     starting the monitor services.

   * The monitor services require deployment-specific configuration to run as
     real daemons. SRU verification should distinguish expected configuration
     errors from import failures. The regression this SRU fixes is at the
     package/import boundary, before service configuration is evaluated.

  [ Other Info ]

  On resolute with masakari-monitors 21.0.0-0ubuntu1, the host monitor
  fails immediately on startup:

  Traceback (most recent call last):
    File "/usr/bin/masakari-hostmonitor", line 6, in <module>
      from masakarimonitors.cmd.hostmonitor import main
  ModuleNotFoundError: No module named 'masakarimonitors.cmd'

  Static inspection of the source package points to the 21.0.0 metadata
  move to pyproject.toml. The current [tool.setuptools] section lists
  only:

  packages = ["masakarimonitors"]

  With setuptools/pyproject metadata this does not install subpackages
  such as masakarimonitors.cmd, masakarimonitors.hostmonitor, etc. The
  console script is still generated, but its target module is missing
  from the installed Python package.

  Expected result:
  masakari-hostmonitor starts, or at least imports masakarimonitors.cmd.hostmonitor successfully.

  More over, autopkgtests fail with:
  autopkgtest [11:55:53]: test autodep8-python3: set -e ; for py in $(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo "Testing with $py:" ; $py -c "import masakari_monitors; print(masakari_monitors)" ; done
  autopkgtest [11:55:53]: test autodep8-python3: [-----------------------
  Testing with python3.14:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
      import masakari_monitors; print(masakari_monitors)
      ^^^^^^^^^^^^^^^^^^^^^^^^
  ModuleNotFoundError: No module named 'masakari_monitors'

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/gazpacho/+bug/2155240/+subscriptions