[Blueprint servercloud-p-complex-deployment-testing] Automated Complex Server Deployment Testing

[James Page]

Blueprint changed by James Page:

Whiteboard changed:
  Scope: Deployment and Testing of Ubuntu Server to support complex requirements i.e. OpenStack + others
  Coverage:
    - OpenStack
-   - Wiki + Database + Memcache + Load Balancer
-   - Networking File Systems
-   - Samba + LDAP or Kerberos Backend
-   - Backup solutions - bacula?
-   - Ruby on Rails stack
+   - Wiki + Database + Memcache + Load Balancer
+   - Networking File Systems
+   - Samba + LDAP or Kerberos Backend
+   - Backup solutions - bacula?
+   - Ruby on Rails stack
  
  UDS-O - Complex Deployment Scenarios:
- 
  
  Method: orchestra+juju AND cloud+juju
  
  james-page proposed a method for testing juju charms on juju at l.u.c - see
  https://lists.ubuntu.com/archives/juju/2011-October/000837.html
  
- Broadley this involves a new charm when can test other charms - results are then collated and can be reported on.
+ This involves a new charm when can test other charms - results are then collated and can be reported on.
  This is then executed by a framework (TBN) which set's up the juju environment and services, executes the charm testing and then can execute tests on the deployed environment.
  I think this is the right way todo this testing - it can be used with any juju provider so can be applied in both physical and cloud environments.
  
- Frequency: TBC
- 
- Daily Test with current development release
- 
- for OpenStack:
- 
- Daily Test of upstream trunk?
+ Frequency: 
+  - Daily Test with current development release
+  - Triggered when key packages change in released versions of Ubuntu in -proposed?
  
  Questions/Proposals:
  [james-page] We need to verify the list of deployments and get volunteers to develop charms for them where gaps exist.
  
  Work Items:
  Develop juju charm testing framework to support all requirements: TODO
  Writing testing hooks for all charms which we want to test this way: TODO
  Setup OpenStack test lab: TODO
  Identify OpenStack testing requirements on the deployed service: TODO
- 
  
  Stuff left from UDS-O not covered in other specs
  --------------------------------------------------------------------
  
  b) Package Removal Testing:
  
  - install services
  - start services
  - check services
  - stop services
  - uninstall services
  - re-install same services
  - start & re-check services
  
  d) Automated Boot Testing:
  
  Minimal install  - startup and shutdown OK
  Full install - startup and shutdown OK
  Complete upgrade and then re-test.
  
  f) Recovery Testing:
  
  Recovery Mode Testing:

-- 
Automated Complex Server Deployment Testing
https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-complex-deployment-testing