Rule-based preferences - format marker RFC, etc.

[Martin Pool]

On 6/24/08, Ian Clatworthy <ian.clatworthy at internode.on.net> wrote:
> John Arbash Meinel wrote:
>  > John Arbash Meinel has voted tweak.
>
>  > I don't really prefer putting the file in as a versioned file in the
>  > users workspace, but I understand the tradeoffs and I'm will to concede
>  > on this one. I would just make a strong request for a required format
>  > marker at the top of the file. Having one for .bzrignore would have
>  > meant we could have retained proper compatibility across versions.

I still think having the file in the tree is the most straightforward
way for users to edit it and for us to include it in conflict handling
and resolution, diffs, web viewers, etc.

>  I'm in agreement about supporting a format marker at the top of the file.
>  The issue is whether it ought to be required or optional. I'm pretty sure
>  poolie has a strong(?) preference for it to be optional. I'm OK with it
>  being required as long as the text is chosen in such a way that users
>  are unlikely to get it wrong. In other words, I think it needs to be as
>  simple as possible, case insensitive and probably whitespace insensitive.
>  How about the suggestion below?
>
>   # format 1
>
>  In terms of regular expressions, I'm thinking ^#\s*format\s*(\d+)\s*$.
>  I considered putting the word "rules" in there as well but I think that
>  increases the chance of someone typing it incorrectly for no noticeable
>  benefit. (It's inside a file called rules or .bzrrules so I don't think
>  repeating that information buys much.)

Putting explicit version markers on data formats that we produce
internally, together with the test scenario support, has worked pretty
well (though not it's not perfect in every regard).  But I think
putting them in user-created files is a bit different.

If we are going to have a marker that is probably a reasonable syntax.
 It would possibly also be ok to use an ini-style variable setting,
but using the line with a hash has the advantage it can be prepended
to files other than an ini file, and detected in reading just the
first bit.

Whether it's required or optional, we need to consider what will
happen if the format marker is unrecognized, incorrect or missing (in
the case that it's required.)  Broadly, it seems like we would have
the choice to give an error, a warning or no indication; and in the
latter two cases to either interpret the file according to a best
guess as to its format, or not use it at all.  Saying "required" is
not really an answer; we need to say "required otherwise the file is
silently ignored" or "required otherwise commits fail" to really
assess it.

The relevant cases seem to be:

 [0] version marker is just what we expect
 [1] user mistyped or otherwise mangled the marker and it is not valid at all
 [2] version marker is the one from an older version of bazaar than is
now being used, and we can still interpret it under the old rules
 [3] ditto, but we don't support it any more
 [4] the marker is for a newer version of bazaar than is currently
running, so we don't properly know how to interpret the file

Case [4] could easily happen if this is added without an overall
tree/repository format bump, and somebody checks out the tree with an
older client.  Of course it's not going to happen for the one version
transition we are sure will happen, which is the one where this is
first added.

If we don't know how to interpret the file then for safety we should
not try to read it at all and just ignore it.  But that could easily
be worse than understanding some but not all of the file -- eol
support might then be broken.  If there is only one file and it has to
be in either the new or old format it will be a sticky situation: to
get new-format features the user will make the tree hard to use from
old clients.  If an unsupported file causes an error this effect is
even stronger.

I think also we need to have an eye on what kind of evolution might
happen here in future.  For instance if we decide this actually
belongs inside the inventory or in some kind of internal structure
rather than in a user file then this version marker will not
particularly help.  If we add new preferences that can be set in
future, or if plugins look for such preferences, then this is also too
crude to stop clients misunderstanding them.  But it would probably
guard against supporting new types of syntax in this file - new ways
of matching files, or changing from an ini file to something else.

I would suggest that if we want to go from an ini file to something
else we could add a new filename.  If we want new rule syntax then
perhaps we just need to make sure the syntax itself has room to grow.
For example if the section headers were

[glob *.py]

then we can add more in future, and potentially also check whether all
of them are understood by the current version.

As far as making sure all the rules are applied in the way the user
expects: I think generally if they are not working, for the ones
contemplated at present, it will be fairly obvious.  (You'll see the
line endings are all changed etc.)  If people do want them then I'd
come back to having a way for the user to say "this tree should be
used with bzr version >= X, and with plugins Y and Z."  I suppose that
could be done either in the tree itself or in branch.conf.

>  Another option is to use a bzr version # instead of a simple integer.
>  My leaning is against this but I could easily change my mind if someone
>  made a strong case for it.

I think then users would be tempted to insert arbitrary bzr versions
either because that's the one they're using or because they want the
behaviour from that version, and we would probably need to support it.

Including the bzr release in the marker is good when we write it out
as a way to give a hint about what version is needed to access a
repository, but that doesn't really apply here.

>  I guess it would be nice if the pattern chosen was adopted (in the future)
>  for .bzrignore as well?

Honestly I don't really see the problem.  We had one transition a long
time ago; we should have thought more about what the defaults were
going to be before adding it but it's not something we're often
changing.  We have ways to add new syntax if it's needed.

-- 
Martin <http://launchpad.net/~mbp/>