[Bug 1768657] [NEW] Testing Fails due to Power Issue with Manual Power Type

Wed May 2 20:46:47 UTC 2018

Public bug reported:

I have an old machine that does not have any form of IPMI or remote
power control. I created a new machine in MAAS with the "Manual" power
type, expecting that I would still be able to use it with MAAS but that
I would need to manually power the machine on and off.

However, while MAAS successfully performs comissioning, all hardware
tests fail due to "BMC never transitioned from unknown to on.". It
appears that when executing tests, MAAS expects to control the machine's
power but can't - and eventually it times out and the test fails.
Consequently, even though I have manually powered on the machine as soon
as it completes a boot from PXE to the login screen, it shuts down again
as there are no tests to run.

May 02 21:32:08 maas maas.drivers.power.manual[929]: [info] You need to check power state of sfsh84 manually.
May 02 21:32:08 maas maas.drivers.power.manual[929]: [info] You need to power on sfsh84 manually.
May 02 21:32:20 maas maas.drivers.power.manual[929]: [info] You need to check power state of sfsh84 manually.
May 02 21:32:20 maas maas.power[929]: [error] Error changing power state (cycle) of node: nagisa (sfsh84)
May 02 21:32:20 maas maas.node[1089]: [info] nagisa: Status transition from TESTING to FAILED_TESTING
May 02 21:32:20 maas maas.node[1089]: [error] nagisa: Marking node failed: Power cycle for the node failed: Failed talking to node's BMC: Failed to power sfsh84. BMC never transitioned from unknown to on.
May 02 21:32:20 maas sh[927]: 2018-05-02 21:32:20 provisioningserver.rpc.power: [critical] nagisa: Power cycle failed.
May 02 21:32:20 maas sh[927]:         Traceback (most recent call last):
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 459, in callback
May 02 21:32:20 maas sh[927]:             self._startRunCallbacks(result)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
May 02 21:32:20 maas sh[927]:             self._runCallbacks()
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks
May 02 21:32:20 maas sh[927]:             current.result = callback(current.result, *args, **kw)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1442, in gotResult
May 02 21:32:20 maas sh[927]:             _inlineCallbacks(r, g, deferred)
May 02 21:32:20 maas sh[927]:         --- <exception caught here> ---
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
May 02 21:32:20 maas sh[927]:             result = result.throwExceptionIntoGenerator(g)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
May 02 21:32:20 maas sh[927]:             return g.throw(self.type, self.value, self.tb)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/provisioningserver/rpc/power.py", line 287, in change_power_state
May 02 21:32:20 maas sh[927]:             system_id, hostname, power_type, power_change, context)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
May 02 21:32:20 maas sh[927]:             result = result.throwExceptionIntoGenerator(g)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
May 02 21:32:20 maas sh[927]:             return g.throw(self.type, self.value, self.tb)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 326, in cycle
May 02 21:32:20 maas sh[927]:             yield self.perform_power(self.power_on, "on", system_id, context)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
May 02 21:32:20 maas sh[927]:             result = g.send(result)
May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 415, in perform_power
May 02 21:32:20 maas sh[927]:             % (system_id, state, state_desired))
May 02 21:32:20 maas sh[927]:         provisioningserver.drivers.power.PowerError: Failed to power sfsh84. BMC never transitioned from unknown to on.
May 02 21:32:20 maas sh[927]:

** Affects: grub2-signed (Ubuntu)
     Importance: Undecided
         Status: Invalid

** Changed in: grub2-signed (Ubuntu)
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to grub2-signed in Ubuntu.
https://bugs.launchpad.net/bugs/1768657

Title:
  Testing Fails due to Power Issue with Manual Power Type

Status in grub2-signed package in Ubuntu:
  Invalid

Bug description:
  I have an old machine that does not have any form of IPMI or remote
  power control. I created a new machine in MAAS with the "Manual" power
  type, expecting that I would still be able to use it with MAAS but
  that I would need to manually power the machine on and off.

  However, while MAAS successfully performs comissioning, all hardware
  tests fail due to "BMC never transitioned from unknown to on.". It
  appears that when executing tests, MAAS expects to control the
  machine's power but can't - and eventually it times out and the test
  fails. Consequently, even though I have manually powered on the
  machine as soon as it completes a boot from PXE to the login screen,
  it shuts down again as there are no tests to run.

  May 02 21:32:08 maas maas.drivers.power.manual[929]: [info] You need to check power state of sfsh84 manually.
  May 02 21:32:08 maas maas.drivers.power.manual[929]: [info] You need to power on sfsh84 manually.
  May 02 21:32:20 maas maas.drivers.power.manual[929]: [info] You need to check power state of sfsh84 manually.
  May 02 21:32:20 maas maas.power[929]: [error] Error changing power state (cycle) of node: nagisa (sfsh84)
  May 02 21:32:20 maas maas.node[1089]: [info] nagisa: Status transition from TESTING to FAILED_TESTING
  May 02 21:32:20 maas maas.node[1089]: [error] nagisa: Marking node failed: Power cycle for the node failed: Failed talking to node's BMC: Failed to power sfsh84. BMC never transitioned from unknown to on.
  May 02 21:32:20 maas sh[927]: 2018-05-02 21:32:20 provisioningserver.rpc.power: [critical] nagisa: Power cycle failed.
  May 02 21:32:20 maas sh[927]:         Traceback (most recent call last):
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 459, in callback
  May 02 21:32:20 maas sh[927]:             self._startRunCallbacks(result)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
  May 02 21:32:20 maas sh[927]:             self._runCallbacks()
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 653, in _runCallbacks
  May 02 21:32:20 maas sh[927]:             current.result = callback(current.result, *args, **kw)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1442, in gotResult
  May 02 21:32:20 maas sh[927]:             _inlineCallbacks(r, g, deferred)
  May 02 21:32:20 maas sh[927]:         --- <exception caught here> ---
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
  May 02 21:32:20 maas sh[927]:             result = result.throwExceptionIntoGenerator(g)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
  May 02 21:32:20 maas sh[927]:             return g.throw(self.type, self.value, self.tb)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/provisioningserver/rpc/power.py", line 287, in change_power_state
  May 02 21:32:20 maas sh[927]:             system_id, hostname, power_type, power_change, context)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
  May 02 21:32:20 maas sh[927]:             result = result.throwExceptionIntoGenerator(g)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
  May 02 21:32:20 maas sh[927]:             return g.throw(self.type, self.value, self.tb)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 326, in cycle
  May 02 21:32:20 maas sh[927]:             yield self.perform_power(self.power_on, "on", system_id, context)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
  May 02 21:32:20 maas sh[927]:             result = g.send(result)
  May 02 21:32:20 maas sh[927]:           File "/usr/lib/python3/dist-packages/provisioningserver/drivers/power/__init__.py", line 415, in perform_power
  May 02 21:32:20 maas sh[927]:             % (system_id, state, state_desired))
  May 02 21:32:20 maas sh[927]:         provisioningserver.drivers.power.PowerError: Failed to power sfsh84. BMC never transitioned from unknown to on.
  May 02 21:32:20 maas sh[927]:

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub2-signed/+bug/1768657/+subscriptions