[Bug 1805191] Re: juju nova-compute charm occasionally errors due to missing /etc/init

Mon Nov 26 17:40:18 UTC 2018

Moving to charm as this is where the problem is.

** Also affects: charm-nova-compute
   Importance: Undecided
       Status: New

** Changed in: cloud-archive
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1805191

Title:
  juju nova-compute charm occasionally errors due to missing /etc/init

Status in OpenStack nova-compute charm:
  New
Status in Ubuntu Cloud Archive:
  Invalid

Bug description:
  When using Juju to deploy openstack using nova-compute charm revision
  287, approximately 10% of the time, the deployment fails with nova-
  compute/0 going into error state (hook failed: "config-changed".  This
  was specifically tested deploying nova-compute into an LXD container.

  This appears to be because /etc/init is not created due to a race
  condition intermittedly.  Note that logging onto the offending LXD
  container and manually creating the directory (sudo mkdir /etc/init)
  and resolving the charm state is sufficient to continue deployment.

  Extract from /var/log/juju/unit-nova-compute-0.log:

  2018-11-26 17:21:51 DEBUG config-changed ERROR no relation id specified
  2018-11-26 17:21:51 INFO juju-log CPU architecture: x86_64
  2018-11-26 17:21:51 INFO juju-log Loaded template from templates/qemu.conf
  2018-11-26 17:21:51 INFO juju-log Rendering from template: /etc/libvirt/qemu.conf
  2018-11-26 17:21:51 INFO juju-log Wrote template /etc/libvirt/qemu.conf.
  2018-11-26 17:21:51 INFO juju-log Loaded template from templates/qemu-kvm
  2018-11-26 17:21:51 INFO juju-log Rendering from template: /etc/default/qemu-kvm
  2018-11-26 17:21:51 INFO juju-log Wrote template /etc/default/qemu-kvm.
  2018-11-26 17:21:52 INFO juju-log Loaded template from templates/libvirtd.conf
  2018-11-26 17:21:52 INFO juju-log Rendering from template: /etc/libvirt/libvirtd.conf
  2018-11-26 17:21:52 INFO juju-log Wrote template /etc/libvirt/libvirtd.conf.
  2018-11-26 17:21:52 INFO juju-log Loaded template from templates/libvirt-bin
  2018-11-26 17:21:52 INFO juju-log Rendering from template: /etc/default/libvirt-bin
  2018-11-26 17:21:52 INFO juju-log Wrote template /etc/default/libvirt-bin.
  2018-11-26 17:21:52 INFO juju-log Loaded template from templates/libvirt-bin.override
  2018-11-26 17:21:52 INFO juju-log Rendering from template: /etc/init/libvirt-bin.override
  2018-11-26 17:21:52 DEBUG config-changed Traceback (most recent call last):
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/config-changed", line 579, in <module>
  2018-11-26 17:21:52 DEBUG config-changed     main()
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/config-changed", line 572, in main
  2018-11-26 17:21:52 DEBUG config-changed     hooks.execute(sys.argv)
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/core/hookenv.py", line 847, in execute
  2018-11-26 17:21:52 DEBUG config-changed     self._hooks[hook_name]()
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1466, in wrapped_f
  2018-11-26 17:21:52 DEBUG config-changed     restart_functions)
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/core/host.py", line 730, in restart_on_change_helper
  2018-11-26 17:21:52 DEBUG config-changed     r = lambda_f()
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1465, in <lambda>
  2018-11-26 17:21:52 DEBUG config-changed     (lambda: f(*args, **kwargs)), restart_map, stopstart,
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 79, in _harden_inner2
  2018-11-26 17:21:52 DEBUG config-changed     return f(*args, **kwargs)
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/config-changed", line 225, in config_changed
  2018-11-26 17:21:52 DEBUG config-changed     CONFIGS.write_all()
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 334, in write_all
  2018-11-26 17:21:52 DEBUG config-changed     [self.write(k) for k in six.iterkeys(self.templates)]
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 334, in <listcomp>
  2018-11-26 17:21:52 DEBUG config-changed     [self.write(k) for k in six.iterkeys(self.templates)]
  2018-11-26 17:21:52 DEBUG config-changed   File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 325, in write
  2018-11-26 17:21:52 DEBUG config-changed     with open(config_file, 'wb') as out:
  2018-11-26 17:21:52 DEBUG config-changed FileNotFoundError: [Errno 2] No such file or directory: '/etc/init/libvirt-bin.override'

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: nova-compute 2:18.0.1-0ubuntu1~cloud0 [origin: Canonical]
  ProcVersionSignature: Ubuntu 4.15.0-1024.25-gcp 4.15.18
  Uname: Linux 4.15.0-1024-gcp x86_64
  NonfreeKernelModules: iptable_raw br_netfilter kvm_intel openvswitch nsh binfmt_misc veth ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_CHECKSUM xt_comment xt_tcpudp iptable_mangle xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat vxlan ip6_udp_tunnel udp_tunnel bridge stp llc ufs msdos xfs ip6table_filter ip6_tables iptable_filter nls_iso8859_1 kvm input_leds serio_raw irqbypass pvpanic sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd psmouse virtio_net
  ApportVersion: 2.20.9-0ubuntu7.5
  Architecture: amd64
  CrashDB:
   {
                  "impl": "launchpad",
                  "project": "cloud-archive",
                  "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
               }
  Date: Mon Nov 26 17:19:27 2018
  PackageArchitecture: all
  ProcEnviron:
   TERM=xterm-256color
   PATH=(custom, no user)
   LANG=C.UTF-8
   SHELL=/bin/bash
  SourcePackage: nova
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-nova-compute/+bug/1805191/+subscriptions