How to handle 'tag pushing revisions' and stacking

[Jelmer Vernooij]

On Thu, 2011-03-31 at 17:17 +0200, John Arbash Meinel wrote:
> I don't think we've accurately sorted out how pushing all tags and
> stacking should work. I'm tempted to back out the pushing-copies-tags in
> the short term, but I wanted to open it up for discussion.
> 
> For those that haven't followed it, the background is generally
> available here:
>   https://bugs.launchpad.net/bugs/745664
> 
> The summary is that if you have a branch that you are pushing up to
> launchpad:
> 
> 1) The first push works as normal, because there is a special code path
> for 'create and populate a new branch'.
> 
> 2) The second push sees "hey you have a branch there, do you have all
> the revisions for all of your tags"? And then proceeds to upload a
> revision for every tag. It then follows that up by pushing the
> inventories for the parent of every tag you have.
> 
> 3) Third and subsequent pushes don't push up new content, but *do* issue
> a 'get_parent_map' check on the set of tags that you have, because they
> are checking if any of them need to be uploaded.
> 
> My immediate response is that we should revert the "push all referenced
> tags" change, until we have an appropriate response for this.
> 
> Specifically, even if we change the logic to say "tagged revisions
> should be available either in the stacked branch, or in its fallbacks",
> then it would improve (2). However, (3+) all now have to issue a giant
> "get_parent_map" call 2 times. First they would hit the stacked branch,
> to find that nothing is present there, and then they hit the stacked-on
> branch, to find that "ok, nothing has changed since the last time I
> pushed, yes all tags really are still there".
> 
> 
> There are some real performance and amount-of-data transferred issues.
> bzr isn't a very large tree, and has about 151 tags. My 'second push'
> case cost 1.2MB of transfer to copy up the revisions (and their
> associated parent revision inventories) for the tags referenced in bzr.
> (Note that it is always the *complete* inventory for all parents, modulo
> shared chk pages. However, tags age, our oldest tag in bzr is about
> 2006, and there aren't many shared chk pages from 2006 to 2011.)
> 
> Emacs has 1129 tags, and the oldest is from 1995 (and there are a lot
> from 1995). Right now, every stacked branch will end up transferring a
> whole lot of extra content. And even if we fix that, they still use a
> lot of bandwidth to check if the revisions are present.
> 
> Thoughts?
I think it's worth considering that if we want to support colocated
branches in the future we will have to support a similar use case with
multiple colocated branches instead of multiple tags. 

I think Vincent has a point that we could consider only looking at those
tags that we are changing. We could add an option to have bzr push look
really hard for missing revisions; this would also take care of another
open bug about filling in ghosts during push if an option is specified.

Cheers,

Jelmer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <https://lists.ubuntu.com/archives/bazaar/attachments/20110331/d8ff2248/attachment.pgp>