Launchpad import of xrms

[David Allouche]

== For the record ==

Brian is a developer of xrms and asked for a bzr mirror of the CVS on Launchpad. Launchpad fails to import it, so we are debugging and trying to fix the problem.

Brian is not subscribed to this mailing list, so please keep him in the recipients list if you reply to this message.

The import back-end is launchpad-cscvs and can be found here:
https://launchpad.net/products/launchpad-cscvs

It is very rough and not ready for general consumption, but help to improve it is gladly accepted.

== Xrms import data ==

The data produced by the failed import is available there:

http://people.ubuntu.com/~david/xrms-import.tgz (85 MB)

It includes three top level entries:

 * bzrworking: the bzr branch produced by the incomplete import
 * cvsworking: a cvs checkout at the time of the import, it contains CVS/Catalog.sqlite, the database used by cscvs to record cvs log data.
 * cscvs-log: the changeset-aware log generate by cscvs

== Reproducing the failure ==

wget http://people.ubuntu.com/~david/xrms-import.tgz
bzr checkout https://launchpad.net/products/launchpad-cscvs cscvs
tar xzf xrms-import.tgz
cd xrms-import/cvsworking
PYTHONPATH=../../cscvs/modules ../../cscvs/cscvs totla -S MAIN.4166:: ../bzrworking

-S means "generate commit messages based on the CVS commits"
MAIN.4166:: means "everything after the last imported changeset, which is number 4166".


== Diagnostic ==

The discussion we had in the past to diagnose the problem is recorded here:

https://launchpad.net/products/bzr-register/+ticket/1936

The synthetic diagnostic is recorded here:

https://help.launchpad.net/VcsImportRequests#xrms

It appears to be a bug in the changeset generation logic.

== Debugging ==

One way to start attacking the problem would be writing a test case using the log of the problematic file:

david at marvin:xrms-import/cvsworking % cvs rlog xrms/include/adodb/pear/readme.Auth.txt

But then we find that the problem is not happening just when processing the log for this file in isolation, using the test case in the attached patch cscvs-xrms-test.patch.

david at marvin:~/cscvs % bzr patch ~/cscvs-xrms-test.patch
david at marvin:~/cscvs % PYTHONPATH=$PWD/modules python test_all.py testXrms

It looks like the problem is occurring when this log is combined with the log of other files. That means that it's not a bug in the log parsing, but in the changeset generation. The corresponding code lives in modules/CVS/CacheGenerator.py.

We can try finding the smallest dataset that causes the problem:

david at marvin:~/xrms-import % rm -rf bzrworking cvsworking
david at marvin:~/xrms-import % cvs -d :pserver:anonymous at xrms.cvs.sourceforge.net:/cvsroot/xrms/ co -d cvsworking xrms/include/adodb/pear
david at marvin:~/xrms-import % bzr init bzrworking
david at marvin:~/xrms-import % cd cvsworking

Using -D3 to increase the verbosity. -D4 prints glob of debugging from the pserver client and the parser, so let us not use this up front.

david at marvin:xrms-import/cvsworking % PYTHONPATH=../../cscvs/modules ../../cscvs/cscvs -D3 totla -S MAIN.1: ../bzrworking 2>&1 | tee ../totla.log

That works okay.

However, if you go one step up:

david at marvin:~/xrms-import % rm -rf bzrworking cvsworking
david at marvin:~/xrms-import % cvs -d :pserver:anonymous at xrms.cvs.sourceforge.net:/cvsroot/xrms/ co -d cvsworking xrms/include/adodb
david at marvin:~/xrms-import % bzr init bzrworking
david at marvin:~/xrms-import % cd cvsworking
david at marvin:xrms-import/cvsworking % PYTHONPATH=../../cscvs/modules ../../cscvs/cscvs -D3 totla -S MAIN.1: ../bzrworking 2>&1 | tee ../totla.log.2

It breaks.

So the bug is triggered by something in the difference between those two imports. Can you look at the logging for something that looks like a clue of where does this revision readme.Auth.txt[1.1.1.2] comes from?

If you need any more help, I would be happy to answer your questions.

-- 
                                                            -- ddaa

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20061213/01fb4b17/attachment.pgp 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cscvs-xrms-test.patch
Type: text/x-patch
Size: 3040 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20061213/01fb4b17/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20061213/01fb4b17/attachment-0001.pgp