[Blueprint servercloud-p-complex-deployment-testing] Automated Complex Server Deployment Testing

[James Page]

Blueprint changed by James Page:

Whiteboard changed:
- Scope: Deployment and Testing of Ubuntu Server to support complex requirements i.e. OpenStack + others
- Coverage:
-   - OpenStack
-   - Wiki + Database + Memcache + Load Balancer
-   - Networking File Systems
-   - Samba + LDAP or Kerberos Backend
-   - Backup solutions - bacula?
-   - Ruby on Rails stack
+ Summary Objective for Precise:
  
- UDS-O - Complex Deployment Scenarios:
+ 1) OpenStack deployment testing in the Ubuntu QA Lab on hardware
  
- Method: orchestra+juju AND cloud+juju
+ This objective is to get OpenStack deployment testing automated for
+ Precise on hardware using Juju and Orchestra.
  
- james-page proposed a method for testing juju charms on juju at l.u.c - see
- https://lists.ubuntu.com/archives/juju/2011-October/000837.html
+ 2) Other Complex Deployment Testing
  
- This involves a new charm when can test other charms - results are then collated and can be reported on.
- This is then executed by a framework (TBN) which set's up the juju environment and services, executes the charm testing and then can execute tests on the deployed environment.
- I think this is the right way todo this testing - it can be used with any juju provider so can be applied in both physical and cloud environments.
- 
- Frequency:
-  - Daily Test with current development release
-  - Triggered when key packages change in released versions of Ubuntu in -proposed?
- 
- Questions/Proposals:
- [james-page] We need to verify the list of deployments and get volunteers to develop charms for them where gaps exist.
+ The framework developed through this blueprint in conjunction with
+ servercloud-p-juju-charm-testing should also support other complex
+ deployment testing.
  
  Work Items:
- Develop juju charm testing framework to support all requirements: TODO
- Writing testing hooks for all charms which we want to test this way: TODO
- Setup OpenStack test lab: TODO
- Identify OpenStack testing requirements on the deployed service: TODO
- 
- Stuff left from UDS-O not covered in other specs
- --------------------------------------------------------------------
- 
- b) Package Removal Testing:
- 
- - install services
- - start services
- - check services
- - stop services
- - uninstall services
- - re-install same services
- - start & re-check services
- 
- d) Automated Boot Testing:
- 
- Minimal install  - startup and shutdown OK
- Full install - startup and shutdown OK
- Complete upgrade and then re-test.
- 
- f) Recovery Testing:
- 
- Recovery Mode Testing:
- 
- 
- aquette propose a method to automate testing of power infrastructure (UPS, power distribution units and servers power supplies):
- http://bazaar.launchpad.net/~ubuntu-bugcontrol/qa-regression-testing/master/view/257//scripts/test-nut.py
+ [james-page] Develop juju charm testing framework to support this requirement: INPROGRESS
+ [james-page] Setup OpenStack test lab: INPROGRESS

-- 
Automated Complex Server Deployment Testing
https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-complex-deployment-testing