[Bug 2167831] [NEW] ceilometer agents fail to start on Python 3.14: PicklingError on _ConfigFileOpt lambda

[Guillaume Boutry]

Public bug reported:

[Impact]
On Ubuntu 26.04 (Python 3.14) the cotyledon-based ceilometer agents
(ceilometer-agent-compute, ceilometer-agent-central, ceilometer-polling,
ceilometer-agent-notification) fail to start their workers:

  Exception in thread Thread-1 (_child_supervisor_thread):
  ...
  _pickle.PicklingError: Can't pickle local object
  <function _ConfigFileOpt.__init__.<locals>.<lambda> ...>

Python 3.14 changed the default multiprocessing start method on Linux from
'fork' to 'forkserver'. cotyledon's ServiceManager uses
multiprocessing.get_context(), so workers are now started by pickling their
arguments, including the parsed oslo_config.ConfigOpts object. With
oslo.config < 10.6.0 the --config-file option type is an unpicklable local
lambda, so spawning fails and the agent collects/publishes nothing.

[Test Plan]
1. Install ceilometer-agent-compute 2:26.0.0-0ubuntu1 on Ubuntu 26.04.
2. Run:
     sudo -u ceilometer /usr/bin/ceilometer-polling --polling-namespaces compute
   Observe the PicklingError and that no worker starts.
3. Install the fixed package and repeat; the worker starts and the agent
   stays up. journalctl -u ceilometer-agent-compute shows no PicklingError.

[Where problems could occur]
The patch explicitly spawns cotyledon workers with the fork start method,
matching pre-3.14 behaviour. A problem would prevent workers from starting.
The heartbeat multiprocessing.Queue in ceilometer-polling keeps the default
context and was verified to work with the fork-spawned workers.

[Other Info]
Root cause is fixed upstream in oslo.config >= 10.6.0 (picklable ConfigOpts
state snapshots for spawn workers); this patch restores pre-3.14 behaviour
for the released 26.04 package. Equivalent fixes: octavia (LP: #2156391),
nova (LP: #2156932).

** Affects: ceilometer (Ubuntu)
     Importance: Undecided
         Status: Invalid

** Affects: ceilometer (Ubuntu Resolute)
     Importance: Undecided
         Status: New

** Affects: ceilometer (Ubuntu Stonking)
     Importance: Undecided
         Status: Invalid

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceilometer in Ubuntu.
https://bugs.launchpad.net/bugs/2167831

Title:
  ceilometer agents fail to start on Python 3.14: PicklingError on
  _ConfigFileOpt lambda

Status in ceilometer package in Ubuntu:
  Invalid
Status in ceilometer source package in Resolute:
  New
Status in ceilometer source package in Stonking:
  Invalid

Bug description:
  [Impact]
  On Ubuntu 26.04 (Python 3.14) the cotyledon-based ceilometer agents
  (ceilometer-agent-compute, ceilometer-agent-central, ceilometer-polling,
  ceilometer-agent-notification) fail to start their workers:

    Exception in thread Thread-1 (_child_supervisor_thread):
    ...
    _pickle.PicklingError: Can't pickle local object
    <function _ConfigFileOpt.__init__.<locals>.<lambda> ...>

  Python 3.14 changed the default multiprocessing start method on Linux from
  'fork' to 'forkserver'. cotyledon's ServiceManager uses
  multiprocessing.get_context(), so workers are now started by pickling their
  arguments, including the parsed oslo_config.ConfigOpts object. With
  oslo.config < 10.6.0 the --config-file option type is an unpicklable local
  lambda, so spawning fails and the agent collects/publishes nothing.

  [Test Plan]
  1. Install ceilometer-agent-compute 2:26.0.0-0ubuntu1 on Ubuntu 26.04.
  2. Run:
       sudo -u ceilometer /usr/bin/ceilometer-polling --polling-namespaces compute
     Observe the PicklingError and that no worker starts.
  3. Install the fixed package and repeat; the worker starts and the agent
     stays up. journalctl -u ceilometer-agent-compute shows no PicklingError.

  [Where problems could occur]
  The patch explicitly spawns cotyledon workers with the fork start method,
  matching pre-3.14 behaviour. A problem would prevent workers from starting.
  The heartbeat multiprocessing.Queue in ceilometer-polling keeps the default
  context and was verified to work with the fork-spawned workers.

  [Other Info]
  Root cause is fixed upstream in oslo.config >= 10.6.0 (picklable ConfigOpts
  state snapshots for spawn workers); this patch restores pre-3.14 behaviour
  for the released 26.04 package. Equivalent fixes: octavia (LP: #2156391),
  nova (LP: #2156932).

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceilometer/+bug/2167831/+subscriptions