[Blueprint servercloud-p-cloud-image-testing] Ubuntu Server Cloud Image Testing

[James Page]

Blueprint changed by James Page:

Whiteboard changed:
- Scope: Official ec2 images for Ubuntu Server across all regions for amd64 and i386
- Coverage:
-   All regions
-   All zones
-   All types
-   Multi-instance
-   Tests:
-     simple-user-data
-     multipart-user-data
-     cloud-config
-     default (no use of metadata service)
+ Summary of Objectives for Precise
  
- Objective:
-  a) Verification that ec2 images boot, reboot and shutdown OK
-  b) Verification that cloud-unit is working OK in images
+ 1) Daily Smoke Testing of Ubuntu AMI's
+ Triggered from cloud image query information.
+ Should cover all images.
+ Check that they boot.
+ Verify each AMI once only.
  
- Method: ubuntu-server-ec2-testing project
- Current Frequency: Candidates for Development Milestones only.
+ 2) Daily AMI testing for the development release
+ Precise should implement daily image testing using basic smoke test to ensure that the AMI is always usable during the development cycle.
  
- Questions/Proposals:
- [james-page] We should be performing a daily test of ec2 images across all regions during development
- [james-page] We published new AMI's for most existing Ubuntu releases - but they don't get verified as good
- [james-page] Reporting of results is currently awkward between jenkins and iso tracker - this needs to be automated/easier
- [james-page] We should test the generic cloud image on OpenStack on a daily basis as well.
+ 3) Daily Testing of Cloud Images
+ In addition to the published AMI's, we should implement testing of the generic cloud images on the following cloud platforms:
+ Eucalyptus
+     Eucalyptus Community Cloud?
+ OpenStack - including KVM and LXC and Xen
+     Canonistack?
+     FreeCloud (NTT and RackSpace)?
  
- Work Items:
+ 4) Full Testing of Release Ubuntu Images
+ Automated testing will be built into the twice weekly image release publishing process and will cover:
+   a) An initial smoke test
+   b) Followed by full image testing
+ Testing will likely be implemented using Jenkins and published to jenkins.qa.ubuntu.com.
+ 
+ Full Notes from UDS-P session: http://pad.ubuntu.com/uds-p-servercloud-p
+ -cloud-image-testing
+ 
+ Work items for precise-alpha-1:
  agree funding of daily AMI testing/cost modelling for justification: TODO
  setup daily testing for current development release: TODO
  setup separate mailing list for test notifications: TODO
+ 
+ Work items for precise-alpha-2:
  [utlemming] update image promotion process to integrate with Jenkins automated testing: TODO
+ update testing framework to make use of spare ec2 time: TODO
+ investigate/implement automated reporting of test results to iso.qa.ubuntu.com: TODO
+ 
+ Work items:
  setup smoke testing of previous ubuntu release images: TODO
- update testing framework to make use of spare ec2 time: TODO
  investigate options for testing ubuntu cloud images on freecloud/canonistack: TODO
  investigate options for testing ubuntu cloud images on eucalyptus: TODO
- investigate/implement automated reporting of test results to iso.qa.ubuntu.com: TODO
  [james blair] freecloud should get full arm servers and integrate: TODO
  consider test of arm disk images or arm partition images in LXC: TODO

-- 
Ubuntu Server Cloud Image Testing
https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-cloud-image-testing