UDD survey results

[Martin Pool]

TLDR? Mixed but generally positive feedback.  Top issues to fix are
speed of branching/merging from Launchpad; keeping import branches
reliably up to date; getting branches where possible to current
formats; removing various small-medium roadblocks; supporting v3
packages and being smarter about merging.

We did a survey before UDS asking people if they had tried using
Bazaar to deal with packaging branches for Ubuntu.   The results are
quite interesting.  Obviously like any opt-in survey there is going to
be some sampling bias, but nevertheless this gives us something to
steer our future efforts.

Most were quite positive, some with constructive criticism; some do
not like the idea at all.

I've put a detailed view survey results up here:

  http://people.canonical.com/~mbp/udd-survey-201011/SurveySummary.html or
  http://people.canonical.com/~mbp/udd-survey-201011/SurveySummary.pdf
(all on one page but unfortunately a bit mangled)

This doesn't have individual sheets, personally identifiable
information and I don't think it has any sensitive information.  It
does have the email addresses for people who chose to leave them, but
not connected to their responses.  Since all of those people have used
their addresses on public lists, I'm going to assume they are ok with
having them on the web.  (If you hear of any problems please let me
know and we'll take it down.)  If anyone wants to drill into
particular responses I will give you access.

85 people answered in all.  I'm grateful to all of them for giving their time.

75 have already used bzr branches for Ubuntu development to some extent.

There is a good even mix of core devs, motus, contributing developers,
casual developers, and new developers.

Net promoter score: 22 would recommend overall "Ubuntu development
using Bazaar" at least fairly strongly (net promoter score 7..10); 6
would recommend avoiding it (0..3).  However, 50 people skipped this
question, perhaps suggesting they have mixed feelings, or the question
was poorly stated (eg they don't see it as a thing they recommend to
others.)

Overall speed: 32 "fast enough", 12 "not fast enough", 41 no answer.
Despite those numbers I think many of the "no answers" actually do see
speed as a major issue, taking the survey as a whole.  The great
majority of comments about speed are to do with fetching from
Launchpad.

Reliability:
Similarly 34 "reliable", 10 "unreliable", 41 no comment, but here my
interpretation is that improving reliability is important, but it is
not at the moment a showstopper the way speed can be.  Some people see
bzr as reliable and lp as flaky; others just the opposite.  lp
downtime is hated, so I'm happy to know this is moving very fast
towards being reduced.  Getting bugfixes in bzr rolled out into the
distro packages is important.  The biggest issue is making sure that
branches are reliably up to date.

The strongest cluster of criticism was essentially "it's not git",
with those respondents saying: they and their potential contributors
are already familiar with git; upstreams and Debian use git for the
package; there are features they like in it; they like git-style
collocated branches; it is faster.  Some of these people would
generally rather we switch to using git, some want Launchpad to
support git alongside bzr, and some want to continue along the line of
interoperation but to do it better.  Some specific features they miss
are mentioned below.  People only made this argument about git: nobody
said we should use support hg or svn alongside or instead of bzr.

Contra this, some people reported bzr easier to set up, learn and use
than git; better error messages.  There were mixed opinions on the
quality of git imports.

Positive things (for at least some people):
* Getting the code, branching and committing reported to be easy.
* bzr itself is nice.
* Having bzr-level log for changes.
* Easily being able to get old release sources.
* Pulling commit messages from changelogs.
* merge-upstream is nice.
* Familiarity of commands from cvs, svn or darcs.
* "When Launchpad is obvious, it works well."
* Pipelines are cool.
* Bugs are generally fixed fast.
* Supportive community.

Criticisms mostly of Launchpad:
* Slow, in various ways, especially from APAC.
* Hard to navigate: finding specific branches; understanding what to
do; hard to find the cool features.
* Confusion/annoyance from old branch formats.
* Bad that the launchpad-side setup of stacked branches is
inconsistent with the client side repository directory.
* When something goes wrong, errors can be confusing.
* "My biggest gripes are with the lp:ubuntu/<pkgname> full source
branches. They have some nice features and traits, but a lot of
problems still: - They are way too easy to break for new upstream
versions from maintainers who aren't used to it yet. - It's impossible
(at least for me) to convert an already existing branch (derived from
upstream bzr) into the "official" lp:ubuntu/<pkgname> branch (ask me
(pitti) for details) - Because of the above, people keep sending MPs
against the wrong branch."
* Not as slick as github.
* "Too clicky", maybe meaning more things should be able to be done
from the command line, or with fewer page loads?
* "Excruciatingly slow" to get feedback from a dput into a ppa.

Criticism/suggestions mostly in bzr:
* Faster branch/merge/pull from Launchpad!  (Either making it directly
faster, allowing local mirrors, or doing a shallow checkout type
thing.)
* Crossing repository formats is annoying.
* Performance, apparently primarily network performance?
* Some(?) merges don't work well.
* Hard to deal with conflicts.
* (I think they mean) 2a formats are difficult when working with old
Debian systems.
* revisionspecs not discoverable.
* Not enough progress-type feedback.
* Should be able to resume interrupted download.
* Important that bug fixes get packaged into the devel series and -updates.
* "The patch format from bzr is awkward" - I'm not sure what this
means; maybe that it is not smart about debdiff stuff
* One user found they needed "regular check/reconcile cycles with
bzr", which isn't something I've heard about in bugs for a long time.
Unfortunately this person did not leave an address.

Criticisms of the workflow, bzr-builddeb, or the integration of udd:
* Imports of packages or branches can be silently outdated.  One user
always checks out-of-band using eg rmadison.
* No relation between upload and commit (then, why commit ?)
* Old tools are familiar.
* "dealing with ignoring files, branch diversions, extracting useful
patch files for upstream work, trunk merges" (bit cryptic)
* Hard to send patches back to debian.
* Yet another layer on top of the packaging toolchain, with new jargon.
* Not as easy as MoM.
* Can't use a mirror, therefore need to fetch everything from London.
* bzr merge-upstream ../trunk/mytarbal-0.0.1.tar.gz ../trunk -r
tag:0.0.1 --version 0.0.1 Notice the replication of "0.0.1"
* Needs better, more stable, git integration.
* Having to rebase branches because of the system getting confused and
causing them to diverge.
* Keeping track of all the branches.
* Better consistent documentation.
* needs

Things that exist but aren't integrated and on by default in the bzr
client, or aren't easy to discover:
* pager
* colored output
* commit --show-diff
* interactive commit
* rewrite and history editing
* generally making plugins more discoverable

Bottom line:
*Heaps* to do, but some encouraging feedback.  The priorities I draw
from this are that we should work on

 * speed of branching/merging from Launchpad
 * keeping import branches reliably up to date
 * removing small-medium disconcerting bugs

then
 * getting branches to current formats, without orphaning old clients
 * supporting v3 packages

this is fairly consistent with what was said at UDS-N, which is nice.

Comments very welcome, and congratulations if you read the whole thing!

-- 
Martin