2.0 upgrade experiences
nmb at wartburg.edu
Wed Aug 12 15:19:16 BST 2009
On 2009-08-12 05:49 , Martin Pool wrote:
> 2009/8/12 Robert Collins<robert.collins at canonical.com>:
>>> 2. The space savings is not as big as my wildest dreams (This is a
>>> standalone branch, even though the parent directory is called "repos".)
>>> nmb at rameses[~/repos/career]$ du -sk .
>>> 6848 .
>>> nmb at rameses[~/repos/career]$ du -sk ../../repos.bak/career
>>> 7028 ../../repos.bak/career
>> You will have content in .bzr/repository/obsolete-packs that will get
>> cleaned up when you make subsequent commits. We do this to deal with
>> various edge cases without requiring fsync/fdatasync operations (which
>> are either a) expensive, or b) [on XFS and MacOSX] don't work).
> I'd like to improve this by
> 1- Making the pack UI say something like this:
> Packing repository ..../career
> 5454MB in 12 packs....
> Thinking hard.....
> Now 1230MB in 1 pack, and 5454MB in 12 obsolete packs that will be
> deleted later
> 2- Adding an option either to pack or to another command to remove
> obsolete packs, and mentioning this from the help for pack. (There is
> a bug for this.)
> The text I propose for #1 is perhaps not ideal, but at least it would
> give a clue that there's a reason why this is happening and it's not
> just something broken, and adding it may be less controversial than
> deleting these files.
> I think if fsync was expensive but reliable and was only run from an
> explicitly requested repack that would be reasonable. However, if
> it's not reliable and there's a chance that a crash near the end of
> packing could lose all your data, we can at least make it more obvious
> why this is happening. I would note though that there's no guarantee
> that the fs will be able to safely read the renamed files.
> The pattern in bug reports seems to be:
> * The space bloat is a real cause for concern.
> * People do suffer fs corruption - either lost files, or files with
> missing blocks. There is a limit to how much we can work around this
> but not deleting files when they're immediately obsolete will probably
> * I think the code that recovers from an unexpectedly-removed pack
> file is working well now, so we don't need the obsolete packs for the
> sake of supporting concurrent clients.
I think that it is important that upgrading to our new, highly-touted
default format should have clear savings in disk space. If fsync is
expensive but reliable, then another option would be to add the
expensive clear-obsolete-packs process to the end of upgrade, which
people already expect to be an expensive process. If reliability cannot
be guaranteed, I think that the current situation is optimal, but needs
better communication about why it's necessary for safety.
>>> 4. The progress bar for upgrade goes 70% of the way to finished
>>> immediately and then takes all of its time working on "Copying content
>>> into repository.:Transferring revisions". Is there any way to make the
>>> progress on the progress bar corrspond more nearly to the time that
>>> various parts of the process actually take?
>> Progress is hard :(. Feel free to file a bug on this, but I don't think
>> that we're likely to get time to really improve it before 2.0 - we have
>> lots of functional issues to fix.
> I don't object to filing a bug but this may be the kind of thing on
> which it's hard to get traction from a bunch of reports for every
> Is the progress bar still spinning there, or counting up, or is it
> just totally still? In other words, is the objection just that
> there's progress but it's not proportional to elapsed time, or that
> there's no progress?
It is spinning (although slowly on my laptop which seems to have to work
quite hard at this process). The only complaint is that the left-right
progress is not proportional to elapsed time.
More information about the bazaar