Any plans/needs to extend the fast-import format?

[Shawn O. Pearce]

Ian Clatworthy <ian.clatworthy at canonical.com> wrote:
> > Actually, one option, "option date-format={raw|rfc2822|now}" is
> 
> Couldn't that simply be ...
> 
> feature date-format=rfc2822

Yes, that's what I was trying to say, but it didn't occur to me
until much later in the email.  :-)
 
> I'd prefer keeping processing options out of the stream altogether. If
> you want a collection of processing options, store them in a separate
> file ala cvs2svn. Each tool can then have a small options file specific
> to it with *suggestions* on how to process it. Users can tweak the
> processing by changing that small file without risking corrupting the
> metadata.

Sverre's patch series started because git is trying to add new
VCS import tools, so Git users can do "git pull" directly from a
bzr or hg repository without needing to use an external separate
command.  That work is based around doing something to the source
to create a fast import stream, and then pipeing that stream into
git-fast-import.  The code seemed to come out a lot cleaner if the
options were passed at the start of the stream rather than through
the command line, as the code that is creating the stream and the
code that is starting up the git-fast-import were a bit far apart
from each other.
 
> To put it another way, I think the fast-import format ought to focus on
> the metadata itself and that the 'feature' feature you suggested is
> enough to achieve that.

Sure, I agree, but the git folks finally talked me into allowing
the git-fast-import command line options at the top of the stream,
and I suspect other tool authors may eventually find something like
that to also be useful.  So we might want to support it, but doing
so does require that the "option" command be tool specific.
 
> BTW, most (but not all) import/export tools support marks files now. The
> internal ID can vary though, e.g. for Bazaar exports, we store
> ':mark revision-id', not ':mark SHA1'.

Interesting, but yes, I did fully expect mark files to be VCS
specific with a VCS revision specificion within them.  One of the
things I regret in the fast-import format, aside from the great
"file order operation" problem, is the fact that the stream format
documents "git SHA1" for revisions when this is clearly not portable.
 
> > Maybe "feature" (see below) is required to be understood, and
> > date-format should be a "feature", while "option" can be skipped only
> > if the VCS name which appears in the option line is not yourself?
> > 
> >   # Must be understood by parser, or parser must abort.
> >   #
> >   feature      ::= 'feature' sp feature_name ('=' path_str)? lf
> >   feature_name ::= path_str
> 
> As Sverre suggested, I think feature-name can be just [A-Za-z_]. I
> didn't think the "= xxx" bit was needed but I've just used it in my
> example above. :-) It could have just as easily been date-format-rfc2822
> though.

Yes, Sverre and I talked about it on IM after I sent the email,
I pointed out to him that using path_str here was a mistake.
Just to paint the bikeshed another color, I would propose
[A-Za-z][A-Za-z0-9-]* instead.

And you are right, we could just as easily do date-format-rfc2822
as we could do date-format=rfc2822, but the latter may be easier
to explain in documentation, e.g. "there should be only zero or
one feature date-format= within a stream".

My personal preference is to allow ('=' path_str)? as I wrote above.
All of us implementors have to have path_str parsing support for
file names, applying it here for values which are most likely safe
as bare words shouldn't be difficult.

-- 
Shawn.