Notes for Friday 2009-11-13

John Arbash Meinel john at arbash-meinel.com
Sun Nov 15 20:01:22 GMT 2009


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is my collection of notes from our Friday discussion. It may be a
bit disjointed, but so was the conversation :).

We spent most of the morning discussing the ideas around 'bzr-builder'
and recipes in general. The basic idea that I took away from it, is that
'bzr-builder' is meant to be a way to take a bunch of branches and
create a working tree. And that 'bzr-builddeb' is meant to be a way to
take that tree and turn it into a .deb package.

There was quite a bit of discussion about how these recipes would be
generated, how they would be stored in launchpad, how they would be
presented to the user, etc.

There was also a good overview from Aaron about how 'bzr-pipelines'
works, and a counterpoint from Robert about 'bzr-loom'. Basically, quite
a bit of overlap in functionality, though loom is more specifically
targeted at being a 'deb' replacement (track a group of patches to an
upstream), versus pipelines being more of a 'get my work done for
review'. Again, similar and some overlap, but quite a few specific
differences based on their different end goals.

John
=:->



bzr-builder recipies

    - Need structured hands-on time
    - Skipped for this sprint

Concrete Next Steps

    - Imports definitely a #2 stage if other things are not #1
    - Build From Branch needs some work to ensure that things are
      generic enough
    - 'bzr-builddeb' has been used more widely
    - 'bzr-builder'
        - probably a step along the ways for 'mini-grumpy'
        - may have more conceptual problems
        - simple cases
            - simplest recipe is 'build from a branch that has
              packaging'
            - Does the recipe exist as a simple text file, or is it
              stored in the database.
        - Being in the DB is concerning. "We expect the data structure
          to need changing, and DB changes take a long time to
          implement."
    - builder is the solution towards getting grumpy
    - Do we want a domain-specific tool, rather than generic things for
      merging X into Y @ Z, monitoring these branches for updates, etc.

    - Difficulties in migrating prototypes to a "production" system
        - Bits were too private, that made it difficult to have people
          using it, and ensuring that it accomplished the goals

    - What does 'BuildFromBranch' look like in Launchpad
        - Having a way of taking an official source package, create a
          trivial 'bzr-builder' recipe, use that build a new package,
          and get that uploaded.
        - The minimal case (not involving a UI in Launchpad) doesn't
          seem to actually provide enough beyond the status quo to
          actually be something better than status quo. And has some
          potential for fragmentation of effort (different people
          getting different experiences trying to build packages.)
        - Start by "push to branch", "trigger rebuild of package",
          "upload to ppa". Once that shows that it looks good, then you
          get distro people asking for it to be used into the main
          archive.

    - Builder for the tree, builddeb for the package
    - What runs on a VM?
    - build-from-branch, etc should be "incrementally adoptable". So it
      doesn't take over the whole toolchain and disrupt the current
      ecosystem

    - 'bzr-builder' can directly help the Ubuntu Distro team as a
      'mini-grumpy' by making it easier for them to get their normal
      builds written.

    - Separating concerns cleanly between what "builder" does and what
      "builddeb" does seems to give a better story. (builder aggregates
      the tree so that you have source + debian, builddeb turns that
      into a .deb package.)

    - Launchpad's role in integrating builder and "build-from-branch"
      is UI, automation and navigation (discovery). Having *us*
      manually using 'bzr-builder' to maintain the 'adopted package
      branches' is a good way to get bzr-builder polished and ensure
      that building a UI on top of it will be beneficial.

    - "build this branch into a ppa" as a trivial Launchpad UI is not
      moving towards having a "recipe" that is being built up.

    - In the LP UI, there will be a "Recipe" page, describing history
      of builds, what branches are involved, what archives the builds
      are going into, etc. When looking at a branch, there should
      potentially be a link to the recipe including it. When looking at
      an archive, it would be good to point back into where it came
      from.

    - What is the long-term UI, what is the short-term UI. A 'text
      area' submit via a form.

    - Latency for a web form can be quite difficult if you don't get
      feeback that your recipe suceeded/failed. If you create the
      recipe locally and test it until it works, and then upload that
      text recipe.

    - We want people-on-netbooks to be able to request Launchpad to do
      all of the heavy lifting. Build-me-a-package so I can upgrade
      this buggy program with this simple fix.

    - Graphs of
        - "how many downloads of a grumpy-built package"
        - "how many trunk series are being built"
        - "number of successful/failing grumpy builds"

    - Shiny "I want to add the branch that fixes bug #2345" to this
      build recipe.

- - What is the Bazaar team going to concretely work on to help w/ Grumpy
    - Operational Quality for supporting the Grumpy effort
    - Maintain the plugins we accepted yesterday, as packages in a ppa.
    - Work on bzr-builder and bzr-builddeb (especially as scratching
      itches for step 1)
    - Maintaining the importer codebases (bzr-git, bzr-hg, bzr-svn)
      (improving code imports)
    - Fixing bugs that LP exposes as part of the build process
    - Improved logging on LP for bzr requests, etc.
    - "Stop-the-line", if something is blocking what you are working
      on, stop and ask a bit to make sure that we know what the sharp
      points are
    - More being reactive about making sure the process is smooth,
      rather than big-design-up-front about what we will do next.
    - Implement "a small patch to launchpad" as a way to engender
      sympathy for lp devs

- - Staging updates
    - Want to get building-from-staging working
    - 'dogfooding.lp.net' getting in the way of getting
      build-from-staging actually working
    - Hardware requirements increasing as we get daily builds automated
      for more packages
    - What is the sysadmin overhead as this rolls out.


- - bzr pipelines

    * Get a branch and checkout to work in
        - bzr cbranch 'frob-repository'
          # Creates a branch 'frob-repository'  and a new checkout of
          # that
        - Alternatively bzr branch --switch $TRUNK ../frob-repository
    * Start working on the repository features
        - bzr commit
        - bzr commit
    * Get to the point of working on ui, start a new pipe
        - bzr add-pipe pipename
            # pipename is evaluated as urlutils.join(branch.base, '..',
            # 'pipename')
        - This preserves any uncommitted changes
        - Creates a new branch, switches the checkout to point to it
        - Points this new branch as following the previous branch
          (linked list, new.prev = old, old.next = new) This is also
          configurable as whether add-pipe puts it before or after the
          current pipe.
    * Do the ui-level changes
        - bzr commit, etc
    * Find you need to do more work in the previous branch
        - bzr switch-pipe :prev
        - Alternatively "bzr switch-pipe named-pipe"
        - This shelves your uncommitted changes into the current pipe,
          and switches you to the target branch
    * Do the new work, commit it.  'pump' the changes up the stack
        - does merge + commit "behind the scenes" into all of the rest
          of the pipeline.
        - If there is a conflict, it switches you to that state, and
          lets you know what happened
    * Switch back to the next pipe, and unshelve your changes in order
    * to
      continue
        - bzr switch-pipe :next
        - bzr unshelve
        - # hack
        - bzr commit
    * 'bzr sync' will push or pull all of the pipes in the pipeline
    * based on
      branch nicks to determine if the branch is logically the same as
      the one in your local pipeline
    * Merge-proposal integration, (push up to launchpad, create a merge
      proposal targeting the development focus with a prerequisite
      branch of the previous branch in the pipeline)

- - The less clean story (post facto)

    * Fix both repository and ui changes (without committing) Switch to
    * the previous pipe (bzr switch-pipe :prev)
      This shelves all changes as well
    * bzr merge --uncommitted :next -i
        - Look in the shelf of the next branch to consider those as
          'uncommitted' changes and allow interactively selecting what
          you want to merge into the local checkout
    * bzr pump bzr switch-pipe :next bzr unshelve
        - since the committed changes were pumped up the unshelve
          results in only the changes that were meant for this pipe

- - bzr-loom discussion

    * Fair amount of overlap between looms and pipelines #1 loom is a
    * different disk format, so you have to have 'bzr-loom'
      installed in order to interact with one of these
    * There is now 'export-loom' which can push all the entries in the
    * loom out
      into regular branches
    * Intended for people maintaining patches against trunk, not as
    * maintaining
      trunk or release series. (trunk as the bottom branch, one
      release-per-thread, or bottom branch = oldest release series, new
      release, bzr up-thread --auto.)
    * 'bzr up-thread' moves you to the next thread, merging in changes
    * you
      don't have yet. "bzr up-thread --auto" takes you to the top,
      merging and committing along the line.
    * Not a finished project, lots of TODO entries key difference is
    * "bzr record" for loom. Pipelines has only the current
      shape-of-the-pipeline, loom tracks the changes of the shape of
      the loom over time
      - Can 'merge' the loom state, and resolve conflicts in the loom
        state
      - If you used loom threads instead of debian/patches, then
        - debian adds a thread (new patch)
        - ubuntu has removed a thread (collapsed a patch)
        - need a way to bring in the mutations from upstream
    * policy issues: when do you record, what does that actually do,
    * how do you
      inspect the history.
    * Separate the branch that the loom was recorded on from the branch
    * that
      I'm hacking on. Currently a bit unclear

- - After-noon discussion topics
    - bzr-builder
        * using it
    - .bzrmeta/*
    - using looms/pipes for 'grumpy'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAksAXhIACgkQJdeBCYSNAANwtwCg0b6ZFWFYEQRrM0VPn4P1FJ0f
oRcAoIEQ8Yuqq+Gnwi0sMfCIj/Ypd+lB
=xtgM
-----END PGP SIGNATURE-----



More information about the ubuntu-distributed-devel mailing list