Notes for Friday 2009-11-13
John Arbash Meinel
john at arbash-meinel.com
Sun Nov 15 20:01:22 GMT 2009
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
This is my collection of notes from our Friday discussion. It may be a
bit disjointed, but so was the conversation :).
We spent most of the morning discussing the ideas around 'bzr-builder'
and recipes in general. The basic idea that I took away from it, is that
'bzr-builder' is meant to be a way to take a bunch of branches and
create a working tree. And that 'bzr-builddeb' is meant to be a way to
take that tree and turn it into a .deb package.
There was quite a bit of discussion about how these recipes would be
generated, how they would be stored in launchpad, how they would be
presented to the user, etc.
There was also a good overview from Aaron about how 'bzr-pipelines'
works, and a counterpoint from Robert about 'bzr-loom'. Basically, quite
a bit of overlap in functionality, though loom is more specifically
targeted at being a 'deb' replacement (track a group of patches to an
upstream), versus pipelines being more of a 'get my work done for
review'. Again, similar and some overlap, but quite a few specific
differences based on their different end goals.
John
=:->
bzr-builder recipies
- Need structured hands-on time
- Skipped for this sprint
Concrete Next Steps
- Imports definitely a #2 stage if other things are not #1
- Build From Branch needs some work to ensure that things are
generic enough
- 'bzr-builddeb' has been used more widely
- 'bzr-builder'
- probably a step along the ways for 'mini-grumpy'
- may have more conceptual problems
- simple cases
- simplest recipe is 'build from a branch that has
packaging'
- Does the recipe exist as a simple text file, or is it
stored in the database.
- Being in the DB is concerning. "We expect the data structure
to need changing, and DB changes take a long time to
implement."
- builder is the solution towards getting grumpy
- Do we want a domain-specific tool, rather than generic things for
merging X into Y @ Z, monitoring these branches for updates, etc.
- Difficulties in migrating prototypes to a "production" system
- Bits were too private, that made it difficult to have people
using it, and ensuring that it accomplished the goals
- What does 'BuildFromBranch' look like in Launchpad
- Having a way of taking an official source package, create a
trivial 'bzr-builder' recipe, use that build a new package,
and get that uploaded.
- The minimal case (not involving a UI in Launchpad) doesn't
seem to actually provide enough beyond the status quo to
actually be something better than status quo. And has some
potential for fragmentation of effort (different people
getting different experiences trying to build packages.)
- Start by "push to branch", "trigger rebuild of package",
"upload to ppa". Once that shows that it looks good, then you
get distro people asking for it to be used into the main
archive.
- Builder for the tree, builddeb for the package
- What runs on a VM?
- build-from-branch, etc should be "incrementally adoptable". So it
doesn't take over the whole toolchain and disrupt the current
ecosystem
- 'bzr-builder' can directly help the Ubuntu Distro team as a
'mini-grumpy' by making it easier for them to get their normal
builds written.
- Separating concerns cleanly between what "builder" does and what
"builddeb" does seems to give a better story. (builder aggregates
the tree so that you have source + debian, builddeb turns that
into a .deb package.)
- Launchpad's role in integrating builder and "build-from-branch"
is UI, automation and navigation (discovery). Having *us*
manually using 'bzr-builder' to maintain the 'adopted package
branches' is a good way to get bzr-builder polished and ensure
that building a UI on top of it will be beneficial.
- "build this branch into a ppa" as a trivial Launchpad UI is not
moving towards having a "recipe" that is being built up.
- In the LP UI, there will be a "Recipe" page, describing history
of builds, what branches are involved, what archives the builds
are going into, etc. When looking at a branch, there should
potentially be a link to the recipe including it. When looking at
an archive, it would be good to point back into where it came
from.
- What is the long-term UI, what is the short-term UI. A 'text
area' submit via a form.
- Latency for a web form can be quite difficult if you don't get
feeback that your recipe suceeded/failed. If you create the
recipe locally and test it until it works, and then upload that
text recipe.
- We want people-on-netbooks to be able to request Launchpad to do
all of the heavy lifting. Build-me-a-package so I can upgrade
this buggy program with this simple fix.
- Graphs of
- "how many downloads of a grumpy-built package"
- "how many trunk series are being built"
- "number of successful/failing grumpy builds"
- Shiny "I want to add the branch that fixes bug #2345" to this
build recipe.
- - What is the Bazaar team going to concretely work on to help w/ Grumpy
- Operational Quality for supporting the Grumpy effort
- Maintain the plugins we accepted yesterday, as packages in a ppa.
- Work on bzr-builder and bzr-builddeb (especially as scratching
itches for step 1)
- Maintaining the importer codebases (bzr-git, bzr-hg, bzr-svn)
(improving code imports)
- Fixing bugs that LP exposes as part of the build process
- Improved logging on LP for bzr requests, etc.
- "Stop-the-line", if something is blocking what you are working
on, stop and ask a bit to make sure that we know what the sharp
points are
- More being reactive about making sure the process is smooth,
rather than big-design-up-front about what we will do next.
- Implement "a small patch to launchpad" as a way to engender
sympathy for lp devs
- - Staging updates
- Want to get building-from-staging working
- 'dogfooding.lp.net' getting in the way of getting
build-from-staging actually working
- Hardware requirements increasing as we get daily builds automated
for more packages
- What is the sysadmin overhead as this rolls out.
- - bzr pipelines
* Get a branch and checkout to work in
- bzr cbranch 'frob-repository'
# Creates a branch 'frob-repository' and a new checkout of
# that
- Alternatively bzr branch --switch $TRUNK ../frob-repository
* Start working on the repository features
- bzr commit
- bzr commit
* Get to the point of working on ui, start a new pipe
- bzr add-pipe pipename
# pipename is evaluated as urlutils.join(branch.base, '..',
# 'pipename')
- This preserves any uncommitted changes
- Creates a new branch, switches the checkout to point to it
- Points this new branch as following the previous branch
(linked list, new.prev = old, old.next = new) This is also
configurable as whether add-pipe puts it before or after the
current pipe.
* Do the ui-level changes
- bzr commit, etc
* Find you need to do more work in the previous branch
- bzr switch-pipe :prev
- Alternatively "bzr switch-pipe named-pipe"
- This shelves your uncommitted changes into the current pipe,
and switches you to the target branch
* Do the new work, commit it. 'pump' the changes up the stack
- does merge + commit "behind the scenes" into all of the rest
of the pipeline.
- If there is a conflict, it switches you to that state, and
lets you know what happened
* Switch back to the next pipe, and unshelve your changes in order
* to
continue
- bzr switch-pipe :next
- bzr unshelve
- # hack
- bzr commit
* 'bzr sync' will push or pull all of the pipes in the pipeline
* based on
branch nicks to determine if the branch is logically the same as
the one in your local pipeline
* Merge-proposal integration, (push up to launchpad, create a merge
proposal targeting the development focus with a prerequisite
branch of the previous branch in the pipeline)
- - The less clean story (post facto)
* Fix both repository and ui changes (without committing) Switch to
* the previous pipe (bzr switch-pipe :prev)
This shelves all changes as well
* bzr merge --uncommitted :next -i
- Look in the shelf of the next branch to consider those as
'uncommitted' changes and allow interactively selecting what
you want to merge into the local checkout
* bzr pump bzr switch-pipe :next bzr unshelve
- since the committed changes were pumped up the unshelve
results in only the changes that were meant for this pipe
- - bzr-loom discussion
* Fair amount of overlap between looms and pipelines #1 loom is a
* different disk format, so you have to have 'bzr-loom'
installed in order to interact with one of these
* There is now 'export-loom' which can push all the entries in the
* loom out
into regular branches
* Intended for people maintaining patches against trunk, not as
* maintaining
trunk or release series. (trunk as the bottom branch, one
release-per-thread, or bottom branch = oldest release series, new
release, bzr up-thread --auto.)
* 'bzr up-thread' moves you to the next thread, merging in changes
* you
don't have yet. "bzr up-thread --auto" takes you to the top,
merging and committing along the line.
* Not a finished project, lots of TODO entries key difference is
* "bzr record" for loom. Pipelines has only the current
shape-of-the-pipeline, loom tracks the changes of the shape of
the loom over time
- Can 'merge' the loom state, and resolve conflicts in the loom
state
- If you used loom threads instead of debian/patches, then
- debian adds a thread (new patch)
- ubuntu has removed a thread (collapsed a patch)
- need a way to bring in the mutations from upstream
* policy issues: when do you record, what does that actually do,
* how do you
inspect the history.
* Separate the branch that the loom was recorded on from the branch
* that
I'm hacking on. Currently a bit unclear
- - After-noon discussion topics
- bzr-builder
* using it
- .bzrmeta/*
- using looms/pipes for 'grumpy'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAksAXhIACgkQJdeBCYSNAANwtwCg0b6ZFWFYEQRrM0VPn4P1FJ0f
oRcAoIEQ8Yuqq+Gnwi0sMfCIj/Ypd+lB
=xtgM
-----END PGP SIGNATURE-----
More information about the ubuntu-distributed-devel
mailing list