2.0 upgrade experiences

[Robert Collins]

On Wed, 2009-08-12 at 20:49 +1000, Martin Pool wrote:


> I'd like to improve this by
> 
> 1- Making the pack UI say something like this:
> 
>   Packing repository ..../career
>   5454MB in 12 packs....
>   Thinking hard.....
>   Now 1230MB in 1 pack, and 5454MB in 12 obsolete packs that will be
> deleted later

This is more explicit about whats going on. It feels like exposing
rather too much of the plumbing to me though :(. Most of our users won't
know:
 - what a pack means here
 - how much later these obsolete packs will be deleted
 - when its safe to delete manually.

SQL databases have a similar thing where temporary data causes disk
space to grow, then shrink - and I haven't yet seen applications that
build on them give me such warnings.

I wonder what it is about our case that makes people so interested and
concerned about this? 

> 2- Adding an option either to pack or to another command to remove
> obsolete packs, and mentioning this from the help for pack.  (There is
> a bug for this.)

I'm ok with this, as its manual and allows users to choose to increase
risk.

> The text I propose for #1 is perhaps not ideal, but at least it would
> give a clue that there's a reason why this is happening and it's not
> just something broken, and adding it may be less controversial than
> deleting these files.
> 
> I think if fsync was expensive but reliable and was only run from an
> explicitly requested repack that would be reasonable.  However, if
> it's not reliable and there's a chance that a crash near the end of
> packing could lose all your data, we can at least make it more obvious
> why this is happening.  I would note though that there's no guarantee
> that the fs will be able to safely read the renamed files.

Situations where we can never really know:
 - remote file systems that don't support fsync().
 - file systems/platforms where fsync() is crippled.

Situations we can know/improve on:
 - local and bzr: connections where the underlying fsync actually works.

> The pattern in bug reports seems to be:
> 
>  * The space bloat is a real cause for concern.
>  * People do suffer fs corruption - either lost files, or files with
> missing blocks.   There is a limit to how much we can work around this
> but not deleting files when they're immediately obsolete will probably
> help.
>  * I think the code that recovers from an unexpectedly-removed pack
> file is working well now, so we don't need the obsolete packs for the
> sake of supporting concurrent clients.

Agreed. Given that we rename on obsolescence, I don't think we've ever
really expected to support concurrent clients via those files. What we
have done is worked on windows via those files - as I recall it windows
supports a rename of an open-but-not-locked file.

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20090813/f1eebfb4/attachment.pgp