Bundling more plugins into bzrtools

[Toshio Kuratomi]

Robert Collins wrote:
> On Wed, 2008-02-13 at 09:15 -0800, Toshio Kuratomi wrote:
>> 1) Many of the plugins don't have a regular release schedule and are 
>> only available as a bzr branch.  For a packager, this means that they 
>> have to poll the upstream branch to check if any significant changes 
>> have occurred since the last snapshot was taken.  If changes have 
>> occurred, they have to create a snapshot of the source.  Then they can 
>> create the updated package.  If the plugins were part of a package that 
>> had regular releases the packager could just subscribe to announcements 
>> for that package or have a script watch for a new tarball to appear in a 
>> download directory and skip directly to step three.
> 
> set -e
> old_revno=$(bzr revno)
> bzr pull
> if [ $(bzr revno) == old_revno ] then;
>     exit
> fi
> new_revno=$(bzr revno)
> bzr export ../$(basename $(pwd))-${new_revno}.tar.bz2
> 
> Throw that in cron, run it as often as you like and now you can have a
> watch script watch that directory.
> 
This only works if upstream has a branch that they are committed to 
keeping stable.  If upstream lacks that methodology then you can't pull 
an arbitrary HEAD from a branch and expect it to work correctly. 
Releases are one method of communicating that a developer thinks a set 
of code is ready for public consumption.  Even though branches are 
actually usable in bzr doesn't mean that everyone is making use of them :-/.

>> 2) Each plugin is a small amount of code but takes practically the same 
>> amount of packaging work as a much larger package.  The overhead for 
>> packaging and maintaining each of bzr-webserve, bzr-bisect, bzr-dbus, 
>> bzr-stats, and etc is the same as maintaining the bzr package itself. 
>> The more packages are consolidated into a single base the more time 
>> downstream maintainers have to spend helping to address actual bugs 
>> rather than looking for new versions of each little package.
> 
> Finding new versions of each package should be /entirely/ automated. If
> its not; I'm happy to provide advice on how to use bzr to do that; and
> for the plugins I write I'm extremely happy to add some small metadata
> to the branches to ease automation.
> 
This piece isn't about finding new versions.  It's about packaging 
separate upstream projects as opposed to packaging a single upstream 
project.  There is overhead for each package created in a distribution 
no matter how small the code is.  So from a packager's perspective, 
twenty individual, small upstream plugins is a much larger time 
commitment than one slightly larger package that contains all of those 
plugins.

>>  From a packager's point of view, the solution to these issues is 
>> upstream consolidation.  If there is an upstream tarball that provides 
>> ten helpful plugins the packager would be able to package 10x as much 
>> functionality.
>>
>> This consolidation is also helpful to the upstream projects.  For 
>> instance, if bzr-dbus, bzr-bisect, and bzr-stats tried to solve problem 
>> #1 individually they would each have to implement build scripts to make 
>> their checkout into a ready to consume tarball
> 
> bzr export if we want to create tarballs; 

Heh, maybe plugins are currently simple enough to just do a bzr export 
to a tarball.

> but really - tarballs are
> slower, are not incrementally updatable, and are generally not very
> convenient. You can't log a tarball, nor update from it. (Well, I hope
> to change this soon, but in the general case).
> 
True, but tarballs are necessary for packaging.  So you aren't going to 
get away from having to generate them for your users even if you have no 
personal need for them.

>> , each have to write 
>> instructions for installing,
> 
> we have to anyway
>
Not really.  bzr-dbus is special enough to warrant a paragraph in the 
README (for the activation server) but do you need to have separate 
installation instructions for bisect and for stats?  Heck, stats and 
bisect don't have installation instructions now, while they're separate.

>>  and each have to create releases when the 
>> time came.
> 
> I don't understand why we'd need to do this; if bzr-dbus is compatible
> with bzr 1.1, but hasn't had any commits, nor any change in
> compatibility with bzr 1.2, why do I need to do a new release of
> bzr-dbus?
> 
You don't.  I said, "when the time came".

>>   If combined, one set of build scripts and one release 
>> manager could do the job for all of them.
>>
>> Aaron Bentley's bzrtools package is a nice illustration of the payoff. 
>> bzrtools has releases that coincide with every bzr major release.  It 
>> has multiple useful plugins that give it great packager bang for the 
>> buck.  Consequently bzrtools is packaged for Debian, Ubuntu, Fedora, 
>> Gentoo, FreeBSD, etc while other useful plugins like stats, bisect, and 
>> dbus are packaged for very few.
> 
> I'm not sure I agree with your cause and effect analysis here. I think
> bzrtools does a much better job of making itself visible to the
> packaging community through releases-every-bzr-release. 

That was one of my causes, yes :-)

The other one, though is even more important.  For a packager to get 
interested in going through the rigamarole of packaging in the first 
place you have to provide enough value that it becomes worthwhile. 
bzrtools provides significant value by bundling so many commands 
together in one upstream package.

> Most of my
> plugins I don't give formal releases to, but I'm happy to write a script
> to do so if thats really the problem.
> 
A formal release with a tarball would certainly be a help as outlined in 
problem #1.  However, it doesn't allow me to scale nearly as much as 
merging with several other plugins (problem #2).  Look at it this way -- 
if bzr-dbus was a part of bzrtools, I probably wouldn't have to do 
anything except add a Requirement on the proper version of dbus to my 
bzrtools package and list the DBus activation file in the proper path. 
If I need to create a new package for bzr-dbus even if it is a released 
tarball I would still have to write a new spec file to control building, 
make sure that the built packages follow the Fedora Guidelines, submit 
the package for review, import it once its reviewed, and kick off a 
separate build for the package.  This is a lot of overhead for two 
commands implemented in five files.

>> Aaron Bentley is also willing to *merge more plugins* into bzrtools if 
>> the plugin authors want that to happen which is the point of this whole 
>> email, really.  Please, plugin authors, let's start a conversation about 
>> merging more of these plugins into bzrtools so your hard work is 
>> packaged up for others to use and thank you for creating :-)
> 
> I'm pretty uninterested in getting my plugins merged into bzrtools. I
> think bzrtools is great, but saying 'you must centralise to get users'
> rather defeats the point of having written the plugin in the first
> place: its meant to be an optional extension, and putting it bzrtools
> will make it non-optional if you want anything else that is already in
> bzrtools.  And thats a problem when, like yesterday, some internal
> changes in the dbus python bindings propogated to hardy and I fixed the
> plugin - but added a stricter dependency. bzrtools has _much_ wider
> compatibility than that.
> 
This is something that packagers have to manage all the time.  Currently 
bzr-gtk has a module that interacts with bzr-dbus.  Fedora doesn't ship 
bzr-dbus because no one wants to go through the trouble of packaging it. 
  Does this mean that bzr-gtk can't be packaged either?  No.  In this 
case it means that that particular module is not installed with our build.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080213/37bd5134/attachment.pgp