Distributed development toolset (Re: ArchiveReorganisation and sponsoring)

[Emmet Hikory]

Daniel Holbach wrote:
> Emmet Hikory schrieb:
>>     I believe this is precisely the point.  For packages that receive
>> 0-1 uploads to Ubuntu per cycle, and for which the Debian version is
>> essentially clean, but it needs some change (e.g. create /var/run/foo
>> at runtime rather than at unpack time) that hasn't been accepted back
>> to Debian, the overhead of a VCS as well as processing the merge and
>> upload seems extra.
>
> With bzr imports of the package's upstream VCS (use one tool to bisect
> and cherry-pick upstream changes and integrate them into the package) it
> will make sense to use bzr even for seldomly uploaded packages.

    Absolutely, and this is a good thing.  In the absence of bzr
imports of the package's upstream VCS, this is less of a benefit.
Where upstream only pushes to the upstream VCS when there is a release
every couple years, even less so.  I very much am not speaking against
bzr or using a VCS, only attempting to show use cases where the
current state of the tools and infrastructure is less useful to
developers, and where there may be perceptions that tracking the
packaging in a VCS may be additional overhead.  From what I know, the
current state is as follows:

1)  Some, but not all packaging is tracked in version control which
developers are expected to keep up to date.
2)  Some, but not all  upstreams are imported into Launchpad for code review
3)  There remain some performance issues in bzr, for various use cases,
4)  There exists significant work towards generating
developer-oriented tools to seamlessly interact with bzr, but this is
not yet complete

I am aware that there are efforts underway to import all revisions
ever seen in Ubuntu into VCS for packaging analysis, but understand
this to be currently blocked on a couple more tweaks to the importer,
support for per-package branch tracking for distributions by
launchpad, and some additional UI thought about how to fit things into
launchpad in an intuitive manner.  This is all worthwhile work, but
until it's complete, it can be complicated to appropriately track
packaging, and so whether it is worth creating an initial branch from
scratch (especially when it is known that a more complete branch will
be available from the revision importer), is very much dependent on
how the developers interested in the package under consideration
prefer to work.

Additionally, launchpad is importing more and more code, on a regular
basis, and ensuring appropriate VCS tracking for an ever-growing
volume of the code used in Ubuntu.  That said, #launchpad has regular
mention of the importer having a slow day, or imports being queued, or
imports needing to be restarted.  This is again a work in progress.
Where upstream code is available in launchpad, it can considerably
improve the experience of cherrypicking to take advantage of that when
managing the packaging.  Where it is not yet available, there is not
yet any benefit.  Similarly to the previous case, whether there is
value in creating branches that track upstream is currently dependent
on the nature of upstream VCS (or the associated LP import), and the
typical activities of the developers interested in that package.

There is significant work being done on bzr performance.  My
experience of working with James to analyse what might be slow in the
case of my pulls of update-manager was that it was possible to
increase the speed of operations by a factor of 8.  I have confidence
that at some point soon, bzr performance for one-time tasks will show
significant improvement, and will be meaningfully comparable to other
tools (whereas in the initial state, my experience was that bzr was
250 times slower than apt-get).  We're not there yet, and so depending
on the latency of a connection, and the likelihood that a given
developer will again change the same package, it may or may not be a
time savings for the developer, and so may or may not be the
appropriate choice.

Lastly, bzr-builddeb is a fast-developing tool, and has seen massive
improvements over the past few cycles, with more expected.  That said,
there are still some known issues with bzr-builddeb, and there are a
number of developers who do use bzr for VCS packaging, and yet do not
use bzr-builddeb.  As bzr-builddeb gets better, and supports more
flexible options for integration with source-only optimised workflows,
it can be expected that a wider number of developers will adopt the
tool.  As this occurs, it will surely become the more common method
recommended to those first becoming involved.  While the advantages of
using bzr VCS packaging are obvious to a bzr-builddeb user, they may
be less so to someone not using this tool, and so again, the decision
of whether to branch and how to branch depends on the developers
working with the packages, and their preferred workflows.

    In summary, there is ongoing work, and the threshhold at which
using bzr for VCS packaging seems the easiest way to work with Ubuntu
packages is approaching, but we're not there yet.  So, rather than
asserting that we should or should not do this, those interested,
either in improving the tools they use daily, or in understanding what
tools others use for comparison, should work with those developing the
tools to improve them.  If the tools are good, and improve things,
this will become obvious as they mature.  If the tools do not improve
things, this will become obvious as some threshold at which adoption
stops is achieved.  As with so many of the other components of the
software stacks we use, these tools ought be judged on their current
state and ability to meet our needs, rather than because they are an
implementation of an exciting technology, or that in an pre-release
state, they happen to meet some of the needs of some of us.
Alternately put, the same tools ought not be discouraged just because
they may feel like additional overhead for some workflows: those
developers experiencing this would do best to either not use the tools
themselves (but leave others the freedom of choice), or work with the
tool developers to improve the tools so that they no longer cause
discomfort.

-- 
Emmet HIKORY