Recipes vs. Looms

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

James Westby wrote:
> Hi all,
> 
> During UDS conversations with various people started to condense some
> thoughts that had been floating in my head for a while around recipes
> and looms: the difference, similarities, overlap and clashes that they
> have. This is another long mail. I apologise for that now, but this is
> a deep topic, and I think it warrants the length, as it is fundamental
> to the things that we plan to do.
> 
> Looms
> =====
> 
> Looms are a different branch format that allow for multiple "threads"
> to exist in a single branch, with a total ordering on them. I believe
> they were conceived with packaging in mind, but either way they are a
> good fit for the way we work.
> 
> For packaging the bottom thread is the upstream code. Above this
> are zero or more threads that modify the code, the "distribution
> patches" as they are now, where the patches become the diff between
> each thread and the one below.
> 
> Above that is the packaging thread that adds the "./debian" directory,
> or at least modifies it with the latest changelog and the like.
> 
> You can imagine changes to this scheme, but that's the basic idea
> that we can work on for now.

A simple example here may help sort things out:

 - debian
 - bugfix 1234
 - bugfix 4567
 - upstream

...

> 
> Looms are also currently missing a way to do the "nest".
> While this may be just a requirement to have nested trees
> in core, there may need to be changes I am not considering.
> 

This isn't entirely true. While there isn't a 'nest' command from looms,
if you create a thread, then do whatever operation necessary to nest
(bzr merge-into, bzr join --by-value, bzr join --by-reference, etc),
then the loom should be able to represent it just fine. going up-thread
will add the nested directory, going down thread would remove it.


> Conclusion
> ==========
> 
> As you can see, there are a fair number of differences, but
> none of them seem intractable.
> 
> There is clearly a disconnect between the two systems though,
> and I fear it will be a mistake to continue down this path.
> 
> Given that recipes don't appear to work very well at all for
> normal Ubuntu development (add a patch type work), and looms
> could be modified to support the daily builds type work
> (update all the threads from their parents and build the
> result), it seems to me as though we should perhaps be looking
> to modify looms, rather than develop a whole new system.
> However I also feel that I am missing something with this
> conclusion, and recipes have more features that perhaps don't
> fit in to the loom model.
> 
> Can anyone see anything I have missed? Does anyone come to
> a different conclusion?
> 
> Thanks,
> 
> James
> 

I think the two systems do have a significant overlap. Recipes are
slightly easier to share, as their state is fairly well defined and
small in scope. Looms *do* have a shareable state, but it isn't quite as
simple as emailing a text file to someone. (Note, though, that if Merge
directives are taught about loom information, you could potentially send
a MD to someone which would build out the loom state.)

I get the feeling that doing your development in looms would be slightly
better. create-thread, hack, up-thread, and your changes are brought
into the integration thread. Versus anything that has to round-trip
through launchpad because you really want to have the official branch
locations in the recipe.

Recipes *might* end up at the same level as debian patch systems. You do
all your work in the actual working dir, and then you generate X and
revert everything you've done to actually get the package built. (X
being a 'diff' in debian patch systems, being a new branch on launchpad
in recipes.)

...

> There are discussions about whether the "stack" model presented
> by loom is the best, as it entagles threads that don't necessarily
> have a dependency. There are alternative models proposed, including
> a full DAG as implemented by TopGit. My fear is that they are
> too complicated for people to easily grasp, and that they are
> not the best way to solve the problems they aim to.
> 

If you make sure the "easy is easy and the hard is possible" it might be
fine. With a decent visualization of the threads and their dependencies,
it does seem a fair amount more powerful. Mostly in that stuff like
"up-thread --auto" wouldn't have to merge everything into everything. It
would also allow for "bzr qlog" at the top thread to be a bit more
obvious as to how the development was meant to be perceived.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAksmsKgACgkQJdeBCYSNAAO4oACgs8DLW2NGvSow2IMo5UWruL+S
Nv4AniusT9EkeTCyTPHMJ2aaf97f1hyr
=lDxd
-----END PGP SIGNATURE-----