Importing Bzr revisions

[David Reitter]

On Mar 23, 2009, at 10:07 AM, Teemu Likonen wrote:

>> I'm just experimenting with "bzr fast-export", which converts to git,
>> and it seems to take about 4 minutes for 1000 revisions on our
>> (modern) server. That would be around 7 hours for my emacs  
>> repository;
>> I can't do that daily.
>>
>> I wonder if there's a way for (bzr) fast-export / (git) fast-import  
>> to
>> work incrementally, i.e. for selected or most recent revisions.
>
> They can or should work incrementally, and actually I have succesfully
> done that. The idea is to use --import-marks and --export-marks  
> options
> with "git fast-import" and --marks option with "bzr fast-export.
>
> I noticed some problems with newer versions of "bzr fast-export",  
> though
> (since it was converted to a proper Bzr command). It seems to corrupt
> the marks file when doing the first incremental export after the  
> initial
> export. At least the revisions are not in right order in the marks  
> file
> anymore. "git fast-import" can't continue to import from the same
> revision where it left last time and it seems to create alternative
> history -- or something.
>
> Really I don't know if this is a bug in Bzr or in Git and haven't
> figured out how to file a useful bug report.



I'm experiencing pretty much the same problem.

Looking at the code (marks_file.py) I don't see why the order would  
matter (even though it would be nicer if the order was consistent).  I  
actually changed this so that it's always sorted, just to help me debug.

Now, I'm getting these errors back from git:

fatal: mark :96985 not declared
fast-import: dumping crash report to .git/fast_import_crash_74262
bzr: broken pipe

I couldn't reproduce this with a simple repository.

However, if one inspects the output of bzr fast-export, one finds  
stuff like this:

commit refs/heads/master
mark :96984
committer <dann> 1237847130 +0000
data 205
....
from :96985
M 644 inline lisp/ChangeLog
data 741178
--- ... --- ... --- ... ---
commit refs/heads/master
mark :96985
committer <jhd> 1237849747 +0000
...
from :96984
M 644 inline src/gtkutil.c
data 135796

I'm not sure about the structure of these files, but my educated guess  
would be that this is a circular reference.
Strange.  One would think that this should never happen.

That said, I get the same errors in other cases as well without  
circular reference.

As an experiment, I deleted the last 1000 or so revisions from the bzr  
marks file, so that they would be output again.
A couple of minutes and 430MB in output later, I imported this on the  
git side, which, after a few seconds, came back with this:
---------------------------------------------------------------------
Alloc'd objects:     105000
Total objects:            5 (      5063 duplicates                  )
       blobs  :            0 (      1549 duplicates          0 deltas)
       trees  :            2 (      2505 duplicates          0 deltas)
       commits:            3 (      1009 duplicates          0 deltas)
       tags   :            0 (         0 duplicates          0 deltas)
Total branches:           1 (         1 loads     )
       marks:        1048576 (     97012 unique    )
       atoms:           1937
Memory total:          5380 KiB
        pools:          2098 KiB
      objects:          3281 KiB
---------------------------------------------------------------------
pack_report: getpagesize()            =       4096
pack_report: core.packedGitWindowSize =   33554432
pack_report: core.packedGitLimit      =  268435456
pack_report: pack_used_ctr            =     104764
pack_report: pack_mmap_calls          =          6
pack_report: pack_open_windows        =          4 /          4
pack_report: pack_mapped              =  100666262 /  100666262
---------------------------------------------------------------------

So, it seems like there were 3 commits that were missing from previous  
transfers.  The final lines of the marks files for bzr and git seem  
coherent.  Hard to identify the culprit.

Further changes resulted in good conversion so far.  Deleting a good  
number of the most recent marks entries seems to be the right thing to  
recover.

A "bzr uncommit" seemed not to make its way to the git side.  No  
good.   I wonder if that is going to create a lasting inconsistency. I  
did get this subsequently:
warning: Not updating refs/heads/master (new tip  
6c81ccc916026d020eadeb0ad6e5b12c18aeccd3 does not contain  
3222cfee5bc00412b6f5e52a420f93564f586ee9)
This "not contained" revision resolved to the uncommitted one - that  
makes sense.
But what consequences does the warning have...?



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2193 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20090325/423c7018/attachment.bin