[Bug 1919451] Re: Update (python-)zhmcclient to latest release

Mon Aug 2 09:51:53 UTC 2021

Thank you Robie for your thorough review and detailed comments &
references.

Thank you Frank for improving upon the original test plan, your
volunteering to help with the verification on real hardware and for your
review and comments regarding the changes.

I also went through the changelog (https://github.com/zhmcclient/python-
zhmcclient/blob/9270ac306586b235751d432cf4a7bd4cfac84c0d/docs/changes.rst)
since 0.22 (i.e. Focal+) that lists a few "incompatible changes" that we
need to be aware of and watch out for:

0.24.0
* Operations that resulted in HTTP status 403, reason 1 ("The user under which the API request was authenticated does not have the required authority to perform the requested action.") so far raised ServerAuthError. However, that exception does not represent that situation properly, because the login user is actually properly authenticated. The handling of this case was changed to now raise HTTPError instead of ServerAuthError. This change is only incompatible to users of the zhmcclient API who have code handling this exception specifically.
=> Tiny API change in the error handling.

0.28.0
* Removed the installed scripts cpcdata and cpcinfo and added them as cpcdata.py and cpcinfo.py to the examples folder.
=> Moving of (example) scripts to a different folder, but they are still available.

0.31.0
* Method 'NotificationReceiver.notifications()' now raises JMS errors returned by the HMC as a new exception 'NotificationJMSError'. JSON parse errors are now raised as a new exception 'NotificationParseError'. Both new exceptions are based on a new base exception 'NotificationError'. (issue #770)
=> Extension of error handling API.

* By default, the zhmcclient now verifies the HMC certificate using the CA certificates in the Python 'certifi' package. This can be controlled with a new 'verify_cert' init parameter to the 'zhmcclient.Session' class. (issue #779)
=> Checking of HMC certificates by default, that can easily be disabled using the 'verify_cert' parameter.

* The 'properties' attribute of the resource classes (e.g. 'Partition') now is an immutable 'DictView' object in order to enforce the stated rule that that callers must not modify the properties dictionary. If your code used to make such modifications nevertheless, it will now get a 'TypeError' or 'AttributeError' exception, dependent on the nature of the modification.
=> This change was reverted in debian/patches/0001-Revert-Changed-properties-attribute-to-be-immutable.patch for compatibility reasons (i.e. missing python immutable-views dependency).

Summary:
- 2 tiny changes in the error handling API
- 1 change of example script installation location
- 1 changed default, that can be controlled via 'verify_cert' parameter
- 1 reverted change that does not affect our users

As Frank stated, the primary user of this package is MAAS v3.0, that was
developed and tested with this newer version in mind already. Apart from
that the use case of zhmc-client is very small with few (if any at all)
external users. Therefore we do not expect to break anybody by
introducing those changes.

I've uploaded the version bump to Hirsute as well, albeit we expect very
little (if any) usage outside the LTS releases. It is included in
Impish/devel already.

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1919451

Title:
  Update (python-)zhmcclient to latest release

Status in Ubuntu on IBM z Systems:
  Fix Committed
Status in zhmcclient package in Ubuntu:
  Fix Released
Status in zhmcclient source package in Focal:
  Incomplete
Status in zhmcclient source package in Hirsute:
  New

Bug description:
  [Impact]

  In order to remotely power-cycle IBM z14 GA2 (or newer) or IBM
  LinuxONE III (or newer) machines/LPARs from within a MAAS controller,
  the python3-zhmcclient libraray (>= v0.29) is needed, to provide all
  the functions required for controlling the HMC.

  [Test Plan]
   * setup a MAAS controller inside a LPAR
   * use that controller to deploy a new Ubuntu Server instance in an additional LPAR
   * use that controller to deploy a new Ubuntu Server instance inside a KVM host (inside another LPAR)
   * Power-cycle the LPAR instance
   * Power-cycle the KVM instance
   * on top and for regression testing the example (as shipped by the zhmcclient) can be ran as well (as far as possible on a non-DPM HMC setup):
  https://github.com/zhmcclient/python-zhmcclient/tree/master/examples

  Also get confirmation from the MAAS team that things are working as
  expected.

  [Where problems could occur]

  This library is s390x only and zhmcclient isn’t widely used yet, it has very limited exposure within the s390 environment. If we would be hit by a regression, we would not be able to power-cycle or deploy any LPAR (or KVM host inside LPAR) machine, using a Focal-based MAAS controller.
  This is basically also the status quo, as the current version in Focal (v0.22) does not allow for that functionality either.

  Reviewing the changes/commits starting from 0.28 to 0.31 (68 + 37 + 27
  + 15 commits) let me think that these are largely unintrusive, just a
  few new features were added (like DPM Capacity Groups), the docs,
  examples and lots of meta updated, error and exception handling
  refined <important>, and last but not least several important bugs
  fixed (for installation, .find for objects <again important>,
  certificates, missing argument in method) - some of them were
  identified during the development of DPM/LPAR support in MAAS earlier
  this year (for example LP#1916475).

  Therefore, and because the zhmcclient is not wide-spread yet (since
  MAAS v3.0 is the first version that can make use of it and was just
  made available on June the 22nd), it's for accessing and managing Z
  systems only via the HMC API, which not many organizations allow (in
  general - for security reasons) and because v0.31 was the version that
  was already used for development _nd final product testing (these days
  as a git clone), makes me rating the regression risk as low.

  [Other Info]
  * https://maas.io/docs/snap/3.0/ui/power-management#heading--configure-use-ibm-z
  * As this is only used on MAAS controllers, running LTS releases, the backport is not needed for Hirsute 21.04
  * The new package dependencies are fulfilled in Focal:
    - python3-requests >= 2.20.1 (focal contains 2.22.0-2ubuntu1 in main)
    - python3-decorator >= 4.0.11 (focal contains 4.4.2-0ubuntu1 in main)

  === Original description ===

  While working on MAAS DPM/LPAR it turned out that a newer python-
  zhmcclient (package zhmcclient) is needed to get certain functions
  work, like especially storage group support.

  The latest is package is v0.27 on hirsute and v0.22 on focal.

  v0.28 is currently the latest release, but 0.29 is coming shortly.
  https://github.com/zhmcclient/python-zhmcclient/issues/731

  Hence an updated package (ideally to 0.29) is needed for focal (knowing that this will require a good justification: https://wiki.ubuntu.com/StableReleaseUpdates).
  Please notice that MAAS was the only reason why the zhmcclient package was created at all.

  The updated package is especially needed for focal, since focal is the
  recommended base distro for MAAS DPM/LPAR, hence this would require a
  special SRU (for this package version bump).

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1919451/+subscriptions