[Bug 2155240] Re: masakari-hostmonitor fails to start: masakarimonitors.cmd not installed

[Guillaume Boutry]

** Description changed:

+ [ Impact ]
+ 
+  * In Resolute, `masakari-monitors` 21.0.0-0ubuntu1 generates the
+    monitor console scripts but installs only the top-level
+    `masakarimonitors` Python package. Subpackages such as
+    `masakarimonitors.cmd`, `masakarimonitors.hostmonitor`,
+    `masakarimonitors.instancemonitor`, and related modules are omitted.
+ 
+  * As a result, `masakari-hostmonitor` fails immediately on startup
+ with:
+ 
+        ModuleNotFoundError: No module named 'masakarimonitors.cmd'
+ 
+    The same package-discovery defect can affect the other Masakari monitor
+    entry points because they also import modules below
+    `masakarimonitors.cmd`.
+ 
+  * This prevents Masakari host monitoring from starting, so deployments that
+    rely on Masakari monitors for host failure detection cannot use the
+    packaged monitor services reliably.
+ 
+  * The upload fixes the package metadata by using setuptools package
+    discovery for `masakarimonitors*`, so all subpackages needed by the
+    generated console scripts are installed. It also corrects the package
+    autopkgtest import check so the test imports the real Python module name,
+    `masakarimonitors`.
+ 
+ [ Test Plan ]
+ 
+  * Use a clean Resolute system or container with the affected package
+    available. Enable the normal Ubuntu archive pocket that contains
+    `masakari-monitors` before testing the pre-update package.
+ 
+  * Reproduce the bug with the affected package:
+ 
+        sudo apt update
+        sudo apt install masakari-host-monitor python3-masakari-monitors
+        python3 -c 'import masakarimonitors; print(masakarimonitors)'
+        python3 -c 'import masakarimonitors.cmd.hostmonitor'
+        masakari-hostmonitor --help
+ 
+    On the affected package, the top-level import may succeed, but importing
+    `masakarimonitors.cmd.hostmonitor` or starting `masakari-hostmonitor`
+    fails with `ModuleNotFoundError: No module named 'masakarimonitors.cmd'`.
+ 
+  * Enable `resolute-proposed`, install the SRU candidate, and verify the
+    package version is the candidate version:
+ 
+        sudo apt update
+        sudo apt install masakari-host-monitor python3-masakari-monitors
+        apt-cache policy masakari-host-monitor python3-masakari-monitors
+ 
+  * Verify that all monitor console-script target modules import:
+ 
+        python3 - <<'PY'
+        import importlib
+ 
+        modules = [
+            "masakarimonitors.cmd.hostmonitor",
+            "masakarimonitors.cmd.instancemonitor",
+            "masakarimonitors.cmd.introspectiveinstancemonitor",
+            "masakarimonitors.cmd.processmonitor",
+        ]
+ 
+        for module in modules:
+            importlib.import_module(module)
+            print(f"OK: {module}")
+        PY
+ 
+  * Verify that the generated entry point reaches option parsing instead of
+    failing during import:
+ 
+        masakari-hostmonitor --help >/tmp/masakari-hostmonitor.help
+        test -s /tmp/masakari-hostmonitor.help
+ 
+  * Run the package autopkgtests for the SRU candidate. The important checks
+    are the Python import check for `masakarimonitors` and the explicit import
+    coverage for the monitor entry-point modules:
+ 
+        autopkgtest masakari-monitors -- schroot resolute-amd64
+ 
+    Use the testbed name appropriate for the SRU verifier's environment.
+ 
+ [ Where problems could occur ]
+ 
+  * The package-discovery change affects which Python modules are installed
+    into `python3-masakari-monitors`. If it accidentally includes too little,
+    one or more monitor services will continue to fail at import time. If it
+    includes unexpected extra files, there could be packaging churn or
+    conflicts with files expected to be owned by a different binary package.
+ 
+  * If the installed subpackages are present but dependency metadata is
+    incomplete, failures may move from `ModuleNotFoundError` for
+    `masakarimonitors.cmd` to import errors for runtime dependencies used by
+    the monitor modules. This would show up when running the import checks or
+    starting the monitor services.
+ 
+  * The monitor services require deployment-specific configuration to run as
+    real daemons. SRU verification should distinguish expected configuration
+    errors from import failures. The regression this SRU fixes is at the
+    package/import boundary, before service configuration is evaluated.
+ 
+ [ Other Info ]
+ 
  On resolute with masakari-monitors 21.0.0-0ubuntu1, the host monitor
  fails immediately on startup:
  
  Traceback (most recent call last):
-   File "/usr/bin/masakari-hostmonitor", line 6, in <module>
-     from masakarimonitors.cmd.hostmonitor import main
+   File "/usr/bin/masakari-hostmonitor", line 6, in <module>
+     from masakarimonitors.cmd.hostmonitor import main
  ModuleNotFoundError: No module named 'masakarimonitors.cmd'
  
  Static inspection of the source package points to the 21.0.0 metadata
  move to pyproject.toml. The current [tool.setuptools] section lists
  only:
  
  packages = ["masakarimonitors"]
  
  With setuptools/pyproject metadata this does not install subpackages
  such as masakarimonitors.cmd, masakarimonitors.hostmonitor, etc. The
  console script is still generated, but its target module is missing from
  the installed Python package.
  
  Expected result:
  masakari-hostmonitor starts, or at least imports masakarimonitors.cmd.hostmonitor successfully.
  
- 
  More over, autopkgtests fail with:
  autopkgtest [11:55:53]: test autodep8-python3: set -e ; for py in $(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo "Testing with $py:" ; $py -c "import masakari_monitors; print(masakari_monitors)" ; done
  autopkgtest [11:55:53]: test autodep8-python3: [-----------------------
  Testing with python3.14:
  Traceback (most recent call last):
-   File "<string>", line 1, in <module>
-     import masakari_monitors; print(masakari_monitors)
-     ^^^^^^^^^^^^^^^^^^^^^^^^
+   File "<string>", line 1, in <module>
+     import masakari_monitors; print(masakari_monitors)
+     ^^^^^^^^^^^^^^^^^^^^^^^^
  ModuleNotFoundError: No module named 'masakari_monitors'

** Changed in: masakari-monitors (Ubuntu Resolute)
       Status: New => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2155240

Title:
  masakari-hostmonitor fails to start: masakarimonitors.cmd not
  installed

Status in Ubuntu Cloud Archive gazpacho series:
  New
Status in masakari-monitors package in Ubuntu:
  Fix Released
Status in masakari-monitors source package in Resolute:
  In Progress
Status in masakari-monitors source package in Stonking:
  Fix Released

Bug description:
  [ Impact ]

   * In Resolute, `masakari-monitors` 21.0.0-0ubuntu1 generates the
     monitor console scripts but installs only the top-level
     `masakarimonitors` Python package. Subpackages such as
     `masakarimonitors.cmd`, `masakarimonitors.hostmonitor`,
     `masakarimonitors.instancemonitor`, and related modules are omitted.

   * As a result, `masakari-hostmonitor` fails immediately on startup
  with:

         ModuleNotFoundError: No module named 'masakarimonitors.cmd'

     The same package-discovery defect can affect the other Masakari monitor
     entry points because they also import modules below
     `masakarimonitors.cmd`.

   * This prevents Masakari host monitoring from starting, so deployments that
     rely on Masakari monitors for host failure detection cannot use the
     packaged monitor services reliably.

   * The upload fixes the package metadata by using setuptools package
     discovery for `masakarimonitors*`, so all subpackages needed by the
     generated console scripts are installed. It also corrects the package
     autopkgtest import check so the test imports the real Python module name,
     `masakarimonitors`.

  [ Test Plan ]

   * Use a clean Resolute system or container with the affected package
     available. Enable the normal Ubuntu archive pocket that contains
     `masakari-monitors` before testing the pre-update package.

   * Reproduce the bug with the affected package:

         sudo apt update
         sudo apt install masakari-host-monitor python3-masakari-monitors
         python3 -c 'import masakarimonitors; print(masakarimonitors)'
         python3 -c 'import masakarimonitors.cmd.hostmonitor'
         masakari-hostmonitor --help

     On the affected package, the top-level import may succeed, but importing
     `masakarimonitors.cmd.hostmonitor` or starting `masakari-hostmonitor`
     fails with `ModuleNotFoundError: No module named 'masakarimonitors.cmd'`.

   * Enable `resolute-proposed`, install the SRU candidate, and verify the
     package version is the candidate version:

         sudo apt update
         sudo apt install masakari-host-monitor python3-masakari-monitors
         apt-cache policy masakari-host-monitor python3-masakari-monitors

   * Verify that all monitor console-script target modules import:

         python3 - <<'PY'
         import importlib

         modules = [
             "masakarimonitors.cmd.hostmonitor",
             "masakarimonitors.cmd.instancemonitor",
             "masakarimonitors.cmd.introspectiveinstancemonitor",
             "masakarimonitors.cmd.processmonitor",
         ]

         for module in modules:
             importlib.import_module(module)
             print(f"OK: {module}")
         PY

   * Verify that the generated entry point reaches option parsing instead of
     failing during import:

         masakari-hostmonitor --help >/tmp/masakari-hostmonitor.help
         test -s /tmp/masakari-hostmonitor.help

   * Run the package autopkgtests for the SRU candidate. The important checks
     are the Python import check for `masakarimonitors` and the explicit import
     coverage for the monitor entry-point modules:

         autopkgtest masakari-monitors -- schroot resolute-amd64

     Use the testbed name appropriate for the SRU verifier's
  environment.

  [ Where problems could occur ]

   * The package-discovery change affects which Python modules are installed
     into `python3-masakari-monitors`. If it accidentally includes too little,
     one or more monitor services will continue to fail at import time. If it
     includes unexpected extra files, there could be packaging churn or
     conflicts with files expected to be owned by a different binary package.

   * If the installed subpackages are present but dependency metadata is
     incomplete, failures may move from `ModuleNotFoundError` for
     `masakarimonitors.cmd` to import errors for runtime dependencies used by
     the monitor modules. This would show up when running the import checks or
     starting the monitor services.

   * The monitor services require deployment-specific configuration to run as
     real daemons. SRU verification should distinguish expected configuration
     errors from import failures. The regression this SRU fixes is at the
     package/import boundary, before service configuration is evaluated.

  [ Other Info ]

  On resolute with masakari-monitors 21.0.0-0ubuntu1, the host monitor
  fails immediately on startup:

  Traceback (most recent call last):
    File "/usr/bin/masakari-hostmonitor", line 6, in <module>
      from masakarimonitors.cmd.hostmonitor import main
  ModuleNotFoundError: No module named 'masakarimonitors.cmd'

  Static inspection of the source package points to the 21.0.0 metadata
  move to pyproject.toml. The current [tool.setuptools] section lists
  only:

  packages = ["masakarimonitors"]

  With setuptools/pyproject metadata this does not install subpackages
  such as masakarimonitors.cmd, masakarimonitors.hostmonitor, etc. The
  console script is still generated, but its target module is missing
  from the installed Python package.

  Expected result:
  masakari-hostmonitor starts, or at least imports masakarimonitors.cmd.hostmonitor successfully.

  More over, autopkgtests fail with:
  autopkgtest [11:55:53]: test autodep8-python3: set -e ; for py in $(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo "Testing with $py:" ; $py -c "import masakari_monitors; print(masakari_monitors)" ; done
  autopkgtest [11:55:53]: test autodep8-python3: [-----------------------
  Testing with python3.14:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
      import masakari_monitors; print(masakari_monitors)
      ^^^^^^^^^^^^^^^^^^^^^^^^
  ModuleNotFoundError: No module named 'masakari_monitors'

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/gazpacho/+bug/2155240/+subscriptions