fast-import / foreign branch interoperability

[Jelmer Vernooij]

Hi Ian,

I'm splitting this out of the original thread, because I think it's
not related to foreign branches per se.

On Sat, Mar 14, 2009 at 11:58:08AM +1000, Ian Clatworthy wrote:
> I'd like foreign branch support and fast-imported repositories to
> play together more nicely. For example, I'd like users to be able
> to use fast-import to generate a mirror repository on which dpush
> could be used. I could be wrong but I think fast-import could
> save the foreign-rev-id -> bazaar rev-id mapping quite easily
> assuming:

> * xxx-fast-export saves the foreign-rev-id -> fast-import-mark mapping
> * bzr-fast-import saves the fast-import-mark -> bzr-rev-id mapping.

> I could easily combine the two to generate a mapping in

>   .bzr/repository/foreign-vcs/rev-id-map

> say. What other mappings does dpush need? One for file-ids as well?

dpush can't use a revision id map at the moment. It currently requires
that the common revisions in the local and remote branch have the same
revision id. The revision id map would not be absolute - it would
provide revision pseudonyms rather than revision aliases, as we can't
be sure that fastimport and bzr-svn/bzr-git create the exact same
revision with the same file ids.

I think the fundamental problem here is parallel imports, since
the foreign branch plugins and fastimport create different revision
ids for (almost the same) revisions. This is also a problem for e.g.
similar revisions imported from a patch independently by different
people, which will result in different revision ids. I think that's
the same problem as we are facing here, and we should look for a
generic solution for that.

> FWIW, I feel it would be uber-nice to have foreign-vcs mapping
> metadata defined in a generic way and stored in a defined location
> like .bzr/repository/foreign-vcs. It ought to be possible to
> generate it *after* the fact by comparing a bzr repository with
> a foreign repository, rather than only at xxx-import time. And
> I've never been a big fan of embedding the data in (potentially
> dozens of) special rev-id namespaces. Among other reasons, it means
> that repository size depends on how compression-efficient the rev-id
> scheme is.
What do you mean by dozens of rev-id namespaces exactly? I don't think
there's any way to get away with changes in the mapping without
changing the revision ids you generate (otherwise we would be
violating referential integrity). bzr foreign branches can't really store
foreign-vcs mapping metadata in .bzr/repository/foreign-vcs since it
may not always have a local repository in which it can store data,
e.g. if you run "bzr log svn://svn.gnome.org/gnome-specimen/trunk".

The revision ids in bzr-svn and bzr-git should be pretty efficient; in
bzr-svn the revision id is usually around 30 characters long, in
bzr-git about 24.

> Looking at the big picture from a high level, I honestly feel that
> the preferred model we ought to encourage is:

> 1. Generate a Bazaar mirror of the foreign repository/branches.
> 2. Bazaar users should treat that as *the* master branch.
> 3. Hooks should be used to keep that master in sync with the foreign master.
This is basically the model that's used by launchpad vcs-imports at the moment.  Among other things, this has the consequence that you're dependent on Launchpad to be able to do proper imports, and that it can't be used for proprietary code. dpush can't work in this particular situation since it needs the revision ids that would be generated by Launchpad, and can't predict them as they are random.

> is preferable to the model of:

> 1. Each user runs import to create their own mirror.
> 2. Different users can merge with each other *if and only if* exactly the
>    same deterministic rev-id (and file-id?) mapping algorithm is used.
The mapping algorithm only changes every so often; there have only
been 2 different versions in the past three years, and one of the
things I hope to work on is making Bazaar deal with different mapping
versions better. 

Cheers,

Jelmer

-- 
Jelmer Vernooij <jelmer at samba.org> - http://jelmer.vernstok.nl/