lock free dirstate - prototype

[Robert Collins]

On Tue, 2009-09-29 at 06:41 +0100, Martin Pool wrote:
> 2009/9/29 Robert Collins <robertc at robertcollins.net>:
> > lp:~lifeless/bzr/dirstate2 has working code.

Updated now, --development-rich-root will use this tree format. No 'bzr
upgrade' support for it yet, so the easiest way to test is' bzr init
--development-rich-root; bzr pull <bzr.dev>' (and you can do that
sensibly in a shared repository).

Some open technical things I'd like folk reviewing it to consider:
 - should we perhaps use a regular lockdir to arbitrate 'changing the
current pointer'. I don't think it makes things safer, but I may be
wrong.
 - how to deal with two writers with the same new content / ABA state
transitions combining with slow-readers that delete what they think is
unlinked content, but actually has been relinked. I'm thinking that slow
readers need to acquire the current file ownership before they unlink
anything.  Two writers with the same new content is harder; one will win
renaming current, and one rolling back will delete the others content. I
think we need some randomness/uniqueness in either the filename or the
content. (e.g. put a pid in there).

> \o/
> 
> Does this write a whole new copy on each update, or incremental files?

Full copy. Theres a bug of stuff we /can/ do in a new tree format; I've
picked a single conceptual bug which has many ramifications and tried to
solve that. Some other ones I've not put any coding effort into are:
 - write less on partial updates
 - only store the left most parent
 - persist an id2path map

I think they are fine things to consider, but all quite orthogonal, and
tricky / time consuming to get right. Specifically, we're not ready in
the rest of the code base to store only the left most parent, we'd be
adding a lot to the size of the data to persist an id2path map, and we'd
need quite careful changes to combine arbitrary numbers of files in a
log structured approach.

All that said, I think we should get this particular aspect I'm working
on polished in a dev setting, and then we can see if there is time and
interest to do more.

> I don't want to gate development or merge of this work into a
> development format, but I do think that before we publicize it we must
> get the formats ui to be better.  The whole issue of understanding
> what format to use, why things are incompatible, or how to upgrade
> them is messy.

Its no better or worse than it was for last few releases. I think we
should improve the formats UI. I'm glad you don't want to make this
merging this improvement conditional on improving the formats UI, but I
really don't understand why telling folk about it should be conditional
on improving the UI *either*.

We have a requirement from Mark to fix the way we manage and announce
development formats, and I think that taking this new format *out of
development* must be gated on that; that is different to the format UI
though - though perhaps that is the actual thing you meant?

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20090929/27531659/attachment.pgp