File format roadmap needed (was Re: [MERGE] New 'development' format.)

[Ian Clatworthy]

Robert Collins wrote:

> I don't want the overhead of three experimental versions, subtree and
> rich-roots can be pulled between (as long as a subtree hasn't been
> used), and anyhow we should be looking at making rich-roots a default
> soon.

Now that 1.0 has shipped, I think we ought to be publishing some
policies w.r.t. file formats and a roadmap reflecting those. I think
this patch is a good start but more is needed, particularly on the
planning front IMO.

While *I* think we've generally done a good job on incrementally
changing formats at the 'right' times, we continue to get flak about how
often we do it (with the implication being that Bazaar is not yet ready
for production use), e.g.
http://www.dribin.org/dave/blog/archives/2007/12/28/dvcs/:

"Also, Bazaar has a history of changing the repository format. Again,
now that Bazaar has hit 1.0, hopefully they won’t be doing this very
often, any more."

So, we either need to:

* do it less often and to a roadmap where (post 1.0) users are excited,
  not annoyed, about each and every upgrade

* make upgrades so painless that users hardly notice.

Being able to change formats easily is almost a strength of ours. I say
"almost" because we're not yet at the point where 'upgrade' can be given
a shared repository and it will magically do each branch using it. Is
that enough to make upgrades painless? Possibly not. We might need a
tool that scans a tree looking for all repositories as well.

One possible policy is "the default file format will not change for N
months" where N is 3, 6 or 12 say. 12 is a good goal but not sensible in
2008.

So if the next change to the default format was 6 months after 1.0, what
"features" would we be targeting for it? Something like this maybe:

* default format good enough for bzr-svn users
* more scalable inventory storage (=> faster commit on big trees)
* stacked repositories (=> history horizons)
* more efficient storage via mpdiffs and/or xdelta?
* other changes required for faster network operations?
* nested trees?

How would that list change if we went with a 3 month window instead?

Ian C.