Reusable fastimport parser

[Greg Ward]

On Mon, Apr 27, 2009 at 7:54 AM, Ian Clatworthy
<ian.clatworthy at internode.on.net> wrote:
> I fully support the idea of pulling out the reusable bits of
> bzr-fastimport. The core parser ought to be pretty stable so I don't see
> it being a problem to occasionally sync against a new version of a core
> (py-fastimport say) library.
>
> I'm yet to look at your code, nor consider the numerous issues, (e.g.
> bzr has a really nice testing framework that bzr-fastimport leverages).
> But I certainly support the concept in principle.

The testing was the only thing that made rearranging bzr-fastimport
remotely interesting.  And it ended up being pretty esay.  The only
code I had to steal from bzrlib.tests was assertEqualDiff() and
_ndiff_strings().  See fastimport/testutils.py in my branch when you
get around to looking at it.

Also, I had to write a runtests.py script.  Not too hard once I
figured out the details of unittest -- something I've been meaning to
do for ages anyways.

> My preference though is to keep the library small and focussed on
> fastimport, rather than bundling it as part of something larger (like
> Dulwich).

Fine by me.  I'm not keen on the mechanics of assigning version
numbers, announcing releases, and all that... so it might be easier to
periodically drop a copy into bzr-fastimport and hg-fastimport and
leave it at that.

> Either way, I'd like Canonical to retain copyright so
> fastimport can be bundled with bzr as a standard plugin once it matures
> enough.

Absolutely.  For me to claim copyright on this would be the height of
arrogance.  I try to stay humble when I first start contributing to a
project, and then crank up the arrogance later, when everyone has been
fooled into thinking I'm a nice guy.  ;-)

> At the other extreme, does it make sense for bzr-fastimport and
> hg-fastimport to be the *one* code base? There's a lot of stuff in
> bzr-fastimport - e.g. fast-import-info, fast-import-info - that isn't
> tied to bzr.

Yes, I've tried to preserve all the interesting-but-nonessential stuff
that isn't tied to Bazaar.

> It might make an interesting experiment in how to write a
> cross-tool plugin! I'm particularly curious about whether we could write
> many of the tests in a cross-tool way.

What a crazy idea.  I'm skeptical that it would work.  For one thing,
I think hg-fastimport should eventually wither away and be folded into
the 'convert' extension shipped with Mercurial.  (I would still like
to have a separate "fastimport" command, for consistency with git and
Bazaar.  But much of Mercurial's conversion infrastructure should be
suitable for fastimport.)

Unified testing is dubious, unless Matt Mackall has a road-to-Damascus
moment about unit tests.  Mercurial mainly uses high-level functional
tests: run this sequence of commands and make sure you get what you're
supposed to get.  Unfortunately, this means there are very few unit
tests in Mercurial.  Something like fastimport really needs both
styles of testing: unit testing at the library level, functional
testing at the plugin level.  (I couldn't help but notice that almost
all of your unit tests are at the library level, i.e. they survived
the transition to my "reusable" library intact.  Unit tests are
fantastic, but I don't think they address everything.)

Greg