SRU MRE Exception for DPDK

Christian Ehrhardt christian.ehrhardt at canonical.com
Fri Dec 15 06:36:31 UTC 2017


Hi Lukasz,
full ack - you just beat my "minion-level moderated reply" to the list
as rbasak has told me the same yesterday.
I'm already prepping it for the SRU Team atm.

Unfortunately while [1] leads to the old Wiki entry which would have
redirected me to the SRU Page.
Instead [2] does only find old MRE requests to the tech board.

[1]: http://lmgtfy.com/?q=Ubuntu+MRE
[2]: http://lmgtfy.com/?q=Ubuntu+minor+release+exception


On Thu, Dec 14, 2017 at 5:46 PM, Lukasz Zemczak
<lukasz.zemczak at canonical.com> wrote:
> Hello Christian!
>
> As per current policy (if I'm not mistaken), MREs for SRUs are now
> handled by the ubuntu-sru team [1], not the technical board. I can
> take a look at that tomorrow - could you draft an Ubuntu wiki page
> with this information in the meantime?
>
> Cheers,
>
> [1] https://wiki.ubuntu.com/StableReleaseUpdates#Documentation_for_Special_Cases
>
> On 14 December 2017 at 10:39, Christian Ehrhardt
> <christian.ehrhardt at canonical.com> wrote:
>> Hi Technical Board
>>
>> I'd like to apply for a Minor Release Exception for the DPDK package.
>>
>> If you want me to join a TB meeting please let me know, but I wanted
>> to give everyone the chance to read into the details before "just
>> showing up on the agenda".
>>
>> Background
>> - ----------
>>
>> DPDK was included in Ubuntu main during the 16.04 release cycle; since
>> then upstream DPDK have started maintaining LTS releases of DPDK;
>> At least for our LTS releases we try to base on DPDK LTS releases
>> The first one was 16.11 which shipped in Zesty, and the next DPDK LTS
>> will be 17.11 which is our target for Ubuntu 18.04 Bionic Beaver.
>>
>> DPDK is a set of libraries and userspace drivers for fast packet
>> processing. It is designed to run on any processors. The first
>> supported CPU was Intel x86 and it is now extended to IBM POWER
>> and ARM. It runs mostly in Linux userland.
>>
>> Having a MRE for DPDK will ensure that users of DPDK receive timely
>> critical updates to this software.
>>
>>
>> Upstream Change and Release Policy
>> - ----------------------------------
>>
>> Upstream have a policy [1] for accepting changes into the LTS release
>> branches which includes:
>>
>>   - Back-porting of any critical bug fixes (crashes, data loss, etc)
>>   - Minor usability items that are very low risk
>>   - Only changes are backported that are part of the last main release
>>     (This ensures more test coverage on those changes)
>>
>> There is a section on backporting features as well, but the
>> constraints limit it to something that is IMHO sane to SRU:
>>
>>   - There is a justifiable use case (for example a new PMD).
>>   - The change is non-invasive.
>>   - There is support within the community.
>>
>> This so far happened very rarely and In addition those features
>> (mostly PMDs to support more HW) are only added in stable
>> releases being not built by default. For packaging that means
>> this is a no-op as we won't enable it so nothing changes.
>>
>> Commits are peer reviewed as part of the normal development process
>> and are signed to signify both the developer and review (see [2] for
>> the doc on this).
>>
>> LTS release updates are made after some time has passed (to allow
>> testing) and usually follow the new master release which happens
>> more or less every 3 months (see [3] for the current road-map).
>>
>> Updates to LTS releases are numbered with a minor point release
>>
>>    16.11: 16.11.1, 16.11.2, ...
>>    17.11: 17.11.1, 17.11.2, ...
>>
>> I watched the DPDK 16.11.x LTS release as it was the first of its
>> kind and it was great. Due to the fast pace of DPDK development with
>> 3 month release cycles a stable release is very important to carry
>> the stability needed by Distribution LTS releases.
>> Therefore I now plan to:
>>
>>   - release Ubuntu 18.04 with the first stable release 17.11.1
>>   - Ask for this MRE to keep up with further stable releases
>>
>>
>> Upstream Regression Testing
>> - ---------------------------
>>
>> The upstream DPDK regression suite is a mix of comprehensive
>> functional tests (API coverage, etc.) and stress workloads via packet
>> generators.  The full set is defined at [4].
>>
>> The QA suite is run against the branches regularly to
>> hunt for low-frequency problems.  Everything should be tested
>> regularly, and all but the most recent patches have been tested over
>> and extended period of time.
>>
>> Results are published via email to the dpdk-test-report mailing list
>> (see [5] for examples).
>>
>> In addition there is a smaller set of integration tests that runs
>> pre-checks. This is integrated into patchwork to directly augment
>> the patch review.
>> Those checks were run by Intel so far but are currently extended
>> to be a Hardware vendor opt-in to gain even more coverage - see [6]
>> for details on this growing part of the project that will provide
>> even more coverage.
>>
>>
>> Ubuntu DPDK Testing
>> - -------------------
>>
>> DPDK has very high constraints on the environment (a lot of memory,
>> huge pages, certain CPU features) as well as the Hardware (limited
>> to a set of cards that have PMDs).
>>
>> Therefore we have a test [7] set outside of autopkgtest that tries to
>> cover the basic use case we know users have in mind (you can
>> use DPDK for way more, but we want and can only test what is in
>> the archive).
>>
>> In particular this sets up a set of KVM guests and runs a few test
>> tools to finally set up a dpdk enabled Open vSwitch between them.
>> On this dpdk enabled Open vSwitch it then runs some benchmarks to
>> ensure no significant performance loss (compared manually for now)
>> and some endurance tests via re-attaching devices or re-starting
>> guests (all those based on lessons learned by past issues we
>> identified).
>>
>> In addition, we set up autopkgtests [8] as well for those components
>> that can be tested. Those are mostly the extra packaging bits that
>> would not be covered by the upstream testing:
>> - testing the init scrips
>> - testing dkms modules work withing Ubuntu
>> - testing the correct linking of builds against dpdk
>>
>> Note: We also test the dpdk autotests in autopkgtests but for the
>> constraints mentioned before they are not reliable in the environment
>> they run in and thereby not yet gating.
>>
>>
>> Proposed SRU Approach
>> - ---------------------
>>
>> SRU updates for DPDK in Ubuntu will be aligned to the associated LTS
>> release of DPDK and only taken care for Ubuntu LTS releases:
>>
>>     18.04 -> DPDK 17.11.x
>>     20.04 -> DPDK 19.11.x (if releases still align)
>>     [...]
>>
>> Ubuntu will only use the released version of updates and will not pull
>> directly from the upstream VCS. That is important as on the release of
>> a DPDK LTS there is another test/verification loop that we want to see
>> passed.
>>
>> Proposed packages will be prepared, uploaded and tested both
>> standalone and in-conjunction with Open vSwitch (following the
>> methodology detail above) as part of the standard SRU verification
>> process for packages with MRE's.
>>
>> I hope this information gives the technical board sufficient
>> confidence that DPDK is worthy of a minor release exception for SRU
>> activity.
>>
>> Thanks
>>
>> Christian
>>
>> [1]: http://dpdk.org/doc/guides/contributing/stable.html
>> [2]: http://dpdk.org/doc/guides/contributing/patches.html
>> [3]: http://dpdk.org/dev/roadmap
>> [4]: http://dpdk.org/doc/dts/gsg/intro.html
>> [5]: http://dpdk.org/ml/archives/test-report/2017-May/020337.html
>> [6]: http://dpdk.org/browse/tools/dpdk-ci/tree/README
>> [7]: https://code.launchpad.net/~ubuntu-server/ubuntu/+source/dpdk-testing/+git/dpdk-testing
>> [8]: http://autopkgtest.ubuntu.com/packages/dpdk
>>
>> --
>> Christian Ehrhardt
>> Software Engineer, Ubuntu Server
>> Canonical Ltd
>>
>> --
>> technical-board mailing list
>> technical-board at lists.ubuntu.com
>> https://lists.ubuntu.com/mailman/listinfo/technical-board
>
>
>
> --
> Ɓukasz 'sil2100' Zemczak
>  Foundations Team
>  lukasz.zemczak at canonical.com
>  www.canonical.com



-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd



More information about the technical-board mailing list