About plainbox-provider-certification-{client,server}

[Jeffrey Lane]

On Mon, Aug 17, 2015 at 7:20 AM Zygmunt Krynicki <
zygmunt.krynicki at canonical.com> wrote:

> Hey
>
> tl;dr; plainbox-provider-certification-{client,server} now live in
> https://code.launchpad.net/plainbox-provider-canonical-certification
> and releases are using git-dpm. Read on for details.
>
> I wanted to explain my actions affecting both certification providers
> that occurred last week. As you know, the last maintenance release did
> happen, but it was made outside of lp:checkbox. This happened due to a
> combination of several factors. You can read about those below. Now
> that it is done, with [1] being merged, let me tell how how the two
> providers are going to live on.
>
> First of all, this is the first time we're going to use git on a
> plainbox provider natively. There might be some learning curve though
> it should be related only to git-on-launchpad and UI issues rather to
> git itself.
>

That's a bit presumptive.  I know of at least one or two people who had and
continue to have a very steep learning curve because of git itself.  Not
everyone is part of the cult of git, nor cares to be and only use git when
absolutely necessary. That's something that should be kept in mind,
especially for those of us who have been using other things exclusively
(BZR) for years and have only seldomly ever touched git.  Couple that with
the weirdness of using LP with git and that learning curve grows steeper
yet.

Not saying this should hamper any progress, however, these sorts of major
paradigm shifts should consider that as well.  I really miss UDS, because
this sort of major change is what would have been discussed and planned and
shared well ahead of things just happening.

Is there some sort of roadmap for these changes that we could track?  Not
sure if your team's trello board is appropriate or not, but if it is, can I
take a peek?

The workflow is conceptually the same as before, where we propose a
> merge request to a given branch an after it lands, it's done. The only
> visible difference here now is that there are two branches
> (master-client and master-server), each holding exactly one of the two
> providers. This is strictly related to simplicity, tagging and
> management of releases. I wanted to move to a model where each
> provider can essentially live in a small repository and everything
> related to the life cycle of that provider would be no different than
> any other provider that follows the same pattern.
>

Makes sense, I was never a fan of having all the providers in one place,
and then all the packaging stuff in a separate place that may or may not
also house other bits of the provider that are non-packaging-related like
canonical-certification.conf.


> The reason there are two branches is very simple, I didn't want to
> split the Launchpad project in two, given how complex and time
> consuming that is. With git it's equally easy to track and work on one
> or one hundred branches, as long as they share one repository. So in
> practice, everything relating to the *client* provider is in the
> master-client branch while the *server* provider is in the
> master-server branch. If you find that confusing we can trivially
> rename that to just client and server, there's no problem in doing
> that.
>

Easy is a relative term.  I've spent 15 minutes now after cloning this
trying to figure out how to get access to the origin-master branch.  It
doesn't seem to exist, or at least I'm not smart enough to figure out how
to access it.

bladernr at sulaco:~/development/git/plainbox-provider-canonical-certification$
git branch -a
* master-client
  remotes/origin/HEAD -> origin/master-client
  remotes/origin/master-client
  remotes/origin/master-server

Again, please do not presume that anyone working on this beyond yourself
knows anything at all about git.  I have yet to sort out how to actually
get this master-server branch.  I would really, really appreciate, for my
sanity and for the sanity of others who have limited or no git experience
or knowledge a useful getting-started doc specifically to cover all these
things that I'm being told are simple, easy and fast.  Because I still find
that anything I do with git, in general, takes me 2 - 3 times as long as
with bazaar and have yet to see any real benefit from all this change, but
that's just my personal opinion, which I've voiced before.

Next thing is merge requests. Apart from the slight UI change (you
> have to target a branch _in_ a repository rather than branch alone)
> everything works as before. I'm sure there will be a few WTF moments
> as we all learn the new Launchpad user interface but we'll get over
> that soon and the knowledge is useful for all the other projects we
> have, or are moving to git. After the merge request is approved a CI
> automaton will merge it. This is still a bit up in the air. I'll work
> with a few people to enhance our CI system to work with git and bzr
> together (i.e. switch to the new "tarmac" prototype). This will happen
> over the next few weeks. We need this for a number of other projects
> that started off with git. This will be no different and I will send
> updates as we progress on this effort.
>

Another layer of complexity for people that are new to all this... at least
in this one, everyone is at the same disadvantage.  I've made one MR for a
git branch so far, and it took me 30 minutes, plus asking and finally
getting an answer on IRC to figure out how to to do so.  I could have done
it the traditional way in less than 5 minutes...

The only thing that *is* significantly different is how any kind of
> "transition" is made. In the past we could have easily made a job
> rename in one commit, that touches all providers at the same time
> (well, except for those that are already out-of-tree, like CDTS). Now
> that will need to be coordinated.


I would suggest the following:

Any "common core" job definitiions exist in plainbox-provider-checkbox.
Any "provider specific" job definitions exist in
plainbox-provider-certification in the relevant branch.  Thus the client
specific jobs like wifi, bluetooth, graphics, etc, would live in the client
branch, and would be maintained and owned by the client team.  Server
specific jobs that client doesn't use, such as multi_nic,
multi-path(coming), large-volume iperf runs, etc live in the server
provider and are maintained by us.

Thus the only things that need to be "coordinated" would be major name
changes to the "Common Core" jobs that apply to both client and server
situations, and the rest of us would be free to change our own provider
specific jobs at our leisure.

I would presume that changes to the common jobs are rare enough that this
would be workable as most/all the job development is on NEW jobs and a name
change is very seldom used.

The only catch there is that we'd need to be better about sharing new job
definitions in case something we create is useful to you and vice-versa.  A
good example of this is TRIM testing on SSDs.  We have a story coming up in
the future to write a test that tests the TRIM function of SSDs because of
a kernel bug discovered where TRIM causes SSDs to end up remounted
read-only.


> This also leads us to the greater
> importance of "API" stability. For a provider, the API is the set of
> jobs it contains. I don't think this is a major issue, we already have
> that problem with our downstream project/commercial derivatives. We
> will just have to tread ourselves the same way as we treat those
> projects. With planning, consideration, tooling and communication.
>
> The last thing, and also the biggest change, is how releases work. The
> key reason I moved away from lp:checkbox was the broken release
> support for this pair providers. Rather than add more complex tooling
> I wanted to removed the excessive complexity that made the tools
> needed in the first place. Releases are simple: they are based on
> signed tags and tarballs (generated with ./manage.py sdist). Tarballs
> are uploaded to Launchpad (to the
> plainbox-provider-canonical-certification project). This is exactly
> how we already treat all of the other providers. We now have less
> divergence as as result of this. This tarball is then combined with a
> packaging branch, like we do in all the Debian and Ubuntu releases
> (though not as we do in our PPAs) and the resulting source package is
> uploaded to a PPA for building.
>
> The major thing here, is that we have now adopted git-dpm. Git DPM is
> the preferred form of modification of all the packages in Debian and
> in Ubuntu. All our Debian packaging will be automatically converted to
> git-dpm in the next few days (or week). This is already done in
> testing but the final trigger will be pulled at around after Debconf.
> This is a Debian-wide change and we are just following the transition.
> Git DPM is *fantastic*. If you ever plan to, or need to, work on a
> release you need to learn this tool. The manual page is a great start.
> There are also several online tutorials but I think there are
> overcomplicated for what this tool is doing. Your mileage may vary.
> Git DPM lets us work, in one tree, with our upstream branches, our PPA
> releases and our Debian releases. I'm sure that we're just getting
> started and as we explore and try we will simplify our release process
> greatly.
>
> There is a downside though, as Launchpad git auto-builder is not yet
> functional we have lost the ability to do daily builds straight from
> the repository. This is not a major change, because this is a project
> that sees less updates than others, but it is a loss non the less. As
> Launchpad evolves this functionality will be restored. For the moment,
> and for our other projects that are also on Git I will be working on
> simple tooling (thanks to git DPM) that lets us upload daily source
> packages from a cron job somewhere.
>

How will this, if at all, affect hot-fixes?  E.g. a bug is discovered in a
test script, fixed and merged?  Will this delay getting that bug fix into a
release or into the daily builds?

The one thing that you must remember from this, is that our packaging
> is no longer based on the tree export from the bzr repository. This is
> a *big* change as now packaging for all kinds of releases (Ubuntu,
> Debian, PPA, daily PPA, etc) for this and all projects that follow the
> same pattern is _identical_. This is a major improvement over our
> prior model, where packaging was totally different an never kept
> up-to-date. The certification provider does not have Debian or Ubuntu
> releases (and likely won't ever have) but the same release tooling
> (git-dpm) applies equally to all of those. I strongly encourage you to
> look at git-dpm, at the packaging branch [2]. I recommend that you
> clone it, and follow the git-dpm manual page to build it. There are
> helpful instructions in the debian/README.Debian file.
>
> If you have any questions about this please don't hesitate to contact me.
>

I understand change is necessary, please understand my concerns are because:

1:  This change to git seemed (to me) to be completely arbitrary and was
never really discussed with anyone outside the client team who also works
on this code
2: More importantly, I have very, VERY limited dev resources on my team,
and a HUGE, almost insurmountable amount of development work left to do for
16.04, in addition to having a large amount of dev work to do on OCP
Checkbox as well.
2a: This forced change to git, for me, at least, means I will be spending
vast more amounts of time trying to use ALL of this new complexity (whether
it is less complex or no, it is a fundamental shift in tools and methods
which have a very steep learning curve) and I do not have that kind of
time, nor resources.  So the net, for me, for now, in the short term, is a
huge time sink.
3:  I'm not blaming anyone, just a bit disappointed in what is a least a
perceived (on my part) lack of communications about major changes to a
shared code base.

Finally, I'm all for change.  SO please take this as just my concerns, I
don't speak for anyone but myself, except for where I see the potential to
delay dev work for my team on new tests and other improvements.  It's
partly frustration, and partly, as I said, because my team and I have very
limited resources and major tooling and packaging shifts eat into those
very limited resources.

I'm sure this is just an immediate growing pain and will ease as time moves
on.

I do appreciate all the very hard work and time you've put into this, and
into explaining in this very descriptive email (as with the previous
announcement about hwcert-tools as well).

Jeff



>
> Best regards
> ZK
>
> [1]
> https://code.launchpad.net/~zyga/checkbox/move-cert-providers-to-git/+merge/268197
> [2]
> https://code.launchpad.net/~checkbox-dev/plainbox-provider-canonical-certification/+git/packaging-client
>
-- 
"Entropy isn't what it used to be."

Jeff Lane - Server Certification Lead, OCP Certification Tools Engineering
Lead,
                  Warrior Poet, Biker, Lover of Pie
Phone: 919-442-8649
Ubuntu Ham: W4KDH                          Freenode IRC: bladernr or
bladernr_
gpg: 1024D/3A14B2DD 8C88 B076 0DD7 B404 1417  C466 4ABD 3635 3A14 B2DD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/checkbox-devel/attachments/20150817/2145c5fb/attachment-0001.html>