[Bug 1581171] Re: pause/resume failing (workload status races)

[Liam Young]

** Changed in: cinder (Juju Charms Collection)
       Status: Fix Committed => Fix Released

** Changed in: glance (Juju Charms Collection)
       Status: Fix Committed => Fix Released

** Changed in: keystone (Juju Charms Collection)
       Status: Fix Committed => Fix Released

** Changed in: ceilometer-agent (Juju Charms Collection)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to cinder in Juju Charms Collection.
Matching subscriptions: charm-bugs
https://bugs.launchpad.net/bugs/1581171

Title:
  pause/resume failing (workload status races)

Status in Landscape Server:
  Fix Committed
Status in Landscape Server 16.06 series:
  Fix Released
Status in ceilometer-agent package in Juju Charms Collection:
  Fix Released
Status in cinder package in Juju Charms Collection:
  Fix Released
Status in glance package in Juju Charms Collection:
  Fix Released
Status in keystone package in Juju Charms Collection:
  Fix Released

Bug description:
  Cinder, keystone, glance, swift-proxy are failing pause/resume amulet
  full tests @ master as of May 10-13+, whereas they were all passing
  prior.

  To date, I've only observed this failure on Trusty targets (ie.
  upstart).  I've not pinned this to any one cause, though it could be
  an init/image revision, or an existing race that is simply exacerbated
  by slower performance during higher load.

  ## Timing observation in one repro, cinder at master
  20:11:38 Test issues pause action
  20:11:49 Test declares failure (based on juju action fetch returning non-zero).
  20:12:22 Services on units get SIGTERM

  ## Juju status
  Shows the charm has declared itself not-paused successfully, which is what the Amulet test goes by:
  http://pastebin.ubuntu.com/16380414/

  
  === original bug description ===
  cinder at master amulet test often fails the pause action test

  Observed and reproduced on trusty-liberty, trusty-mitaka test combos.

  ## Unit status
  cinder/0          blocked        idle        1.25.5  1                10.5.1.105     Services should be paused but these services running: cinder-api, cinder-volume, cinder-scheduler, Paused. Use 'resume' action to resume normal service.

  ## Amulet failure (trusty-mitaka, cinder at master 2016 May 12)
  2016-05-12 18:11:56,004 test_910_pause_and_resume DEBUG: Checking pause and resume actions...
  Running command: juju action do --format=json cinder/0 pause

  Traceback (most recent call last):
    File "/tmp/bundletester-QVbeQC/cinder/tests/gate-basic-trusty-mitaka", line 11, in <module>
      deployment.run_tests()
    File "/tmp/bundletester-QVbeQC/cinder/tests/charmhelpers/contrib/amulet/deployment.py", line 95, in run_tests
      getattr(self, test)()
    File "/tmp/bundletester-QVbeQC/cinder/tests/basic_deployment.py", line 773, in test_910_pause_and_resume
      assert self._wait_on_action(action_id), "Pause action failed."
  AssertionError: Pause action failed.

  PASS: 1 ERROR: 1 Total: 2 (869.865811 sec)

  ## Juju status
  http://pastebin.ubuntu.com/16380414/

To manage notifications about this bug go to:
https://bugs.launchpad.net/landscape/+bug/1581171/+subscriptions