[Bug 1581171] Re: pause/resume failing

[OpenStack Infra]

Reviewed:  https://review.openstack.org/316194
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=13cf8d93500c8bd148641aba0627cce65c0a2a07
Submitter: Jenkins
Branch:    master

commit 13cf8d93500c8bd148641aba0627cce65c0a2a07
Author: David Ames <david.ames at canonical.com>
Date:   Fri May 13 09:43:20 2016 -0700

    Charm-helpers sync to pull in service_running fix
    
    Change-Id: If0ccd1a3c4156fedb45fe5bdec8cc74bc7e0257a
    Closes-Bug: 1581171


** Changed in: cinder (Juju Charms Collection)
       Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to cinder in Juju Charms Collection.
Matching subscriptions: charm-bugs
https://bugs.launchpad.net/bugs/1581171

Title:
  pause/resume failing

Status in cinder package in Juju Charms Collection:
  Fix Committed
Status in glance package in Juju Charms Collection:
  Fix Committed
Status in keystone package in Juju Charms Collection:
  In Progress

Bug description:
  Cinder, keystone, glance, swift-proxy are failing pause/resume amulet
  full tests @ master as of May 10-13+, whereas they were all passing
  prior.

  To date, I've only observed this failure on Trusty targets (ie.
  upstart).  I've not pinned this to any one cause, though it could be
  an init/image revision, or an existing race that is simply exacerbated
  by slower performance during higher load.

  ## Timing observation in one repro, cinder at master
  20:11:38 Test issues pause action
  20:11:49 Test declares failure (based on juju action fetch returning non-zero).
  20:12:22 Services on units get SIGTERM

  ## Juju status
  Shows the charm has declared itself not-paused successfully, which is what the Amulet test goes by:
  http://pastebin.ubuntu.com/16380414/

  
  === original bug description ===
  cinder at master amulet test often fails the pause action test

  Observed and reproduced on trusty-liberty, trusty-mitaka test combos.

  ## Unit status
  cinder/0          blocked        idle        1.25.5  1                10.5.1.105     Services should be paused but these services running: cinder-api, cinder-volume, cinder-scheduler, Paused. Use 'resume' action to resume normal service.

  ## Amulet failure (trusty-mitaka, cinder at master 2016 May 12)
  2016-05-12 18:11:56,004 test_910_pause_and_resume DEBUG: Checking pause and resume actions...
  Running command: juju action do --format=json cinder/0 pause

  Traceback (most recent call last):
    File "/tmp/bundletester-QVbeQC/cinder/tests/gate-basic-trusty-mitaka", line 11, in <module>
      deployment.run_tests()
    File "/tmp/bundletester-QVbeQC/cinder/tests/charmhelpers/contrib/amulet/deployment.py", line 95, in run_tests
      getattr(self, test)()
    File "/tmp/bundletester-QVbeQC/cinder/tests/basic_deployment.py", line 773, in test_910_pause_and_resume
      assert self._wait_on_action(action_id), "Pause action failed."
  AssertionError: Pause action failed.

  PASS: 1 ERROR: 1 Total: 2 (869.865811 sec)

  ## Juju status
  http://pastebin.ubuntu.com/16380414/

To manage notifications about this bug go to:
https://bugs.launchpad.net/charms/+source/cinder/+bug/1581171/+subscriptions