[Blueprint servercloud-q-server-benchmark-and-performance] ARM Server Benchmarking and Performance

[Craig Magina]

Blueprint changed by Craig Magina:

Whiteboard changed:
  I'm deferring this due to no workitems defined. -robbiew
  
  From Etherpad:
  
  Workload tests:
   - bzr branch lp:server-workload-testing
   - Currently done and maintained by Canonical
   - Linaro could help getting it to work across the ARM boards available there as well as with LAVA
  openssl: openssl-test? gnu-tls?
  ARM Workload:
   - Linaro is looking at extracting the power consumed when stressing the board support
   - It'd be useful to benchmark not only the test results, but the power that was consumed during the test run
  Areas to investigate while checking for performance:
   - Not that clear at the moment
   - Generally looking at the usual test cases/benchmarks for a start
   - One way would be to get feedback from the hw vendors to understand what would be the priority
  ACTION:
  [rsalveti] Make sure we have a session at Linaro Connect to cover power usage at the ARM boards during test case executions
  IO/Storage: Ceph, Swift, Gluster?
  Areas of focus on feature/functional side:
   - KVM
   - LXC
   - OpenMPI
  Calxeda is interested in OpenMPI performance/testing.
  From P:
  As  actual ARM server hardware begins to enter the marketplace, a   standardized way to test and benchmark performance must be developed and   run on all active platforms we support for ARM server.
  http://www.phoronix-test-suite.com/
  http://openbenchmarking.org/
  Upcoming PTS tests: MySQL, TPCC MySQL, various SPEC tests (might not  be public), Unixbench, plus others (can also easily add in any others,  just ask)
  https://wiki.ubuntu.com/ARM/QA/Server
  https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-qa
  https://blueprints.launchpad.net/ubuntu/+spec/server-o-arm-image-validation
  - membase?
  - Ensure that multi-system testing setups are standardized
  - weekly meeting to reveiw testing on #ubuntu-meeting (Thursdays 15:00 UTC)
  ACTIONS:
  - Automate testing [explore PTS documentation about batch-run options]
  - Find an IPV6 testing tool
  - Add a test for Membase
  - Add a test for Hadoop and other workloads
  - Explore stress/load testing (memtester, stress, kernel build test, netperf, unix bench)
  - Test on real hardware when available
  - Compare P test results against O.
  - AHCI testing
  - Investigate PTS logging sensor data [Run any test(s) with  "MONITOR=all" environmental variable to see all sensors... or see  included HTML/PDF documentation... or run "phoronix-test-suite system-sensors" to see all supported/unsupported sensors. Try latest PTS release for latest sensor coverage.]
  
  ------
- Information on integrating Blueprint spec integration into the whiteboard is at https://wiki.ubuntu.com/BlueprintSpec
+ =User Stories=
+  - George wants to setup an openmpi cluster on his new ARM server nodes.
+ 
+ =Assumptions=
+  -All of the mentioned pieces of software already work on ARM platforms.
+ 
+ =Test Plans=
+  - Run each script on all available ARM server hardware and some x86 hardware.
+ 
+ =Release Note=
+  - Automated testing of key clustered workloads and core Linux stacks has been performed to ensure a more reliable experience on ARM and x86 hardware.

-- 
ARM Server Benchmarking and Performance
https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-server-benchmark-and-performance