0.6 release plan

[John A Meinel]

Nicholas Nethercote wrote:
> On Wed, 26 Oct 2005, John A Meinel wrote:
> 
>> Well, I have to say that you are correct that python's gzip isn't as
>> fast as the native gzip. Specifically, I tested this:
>>

...

> 
> I wouldn't trust these numbers, for two reasons.  First, they are much
> too small;  less than 0.1s is at the level of noise.  Second, you're
> comparing apples with oranges -- what are the relative overheads of
> starting zcat vs. Python doing its additional stuff?  How much of the
> time in each case is spent actually decompressing?

The point specifically was to look into python's gzip routines, and see
if they are reasonably efficient. I was doing it on a file that we would
actually want to be compressing (inventory.weave).

I wasn't trying to equate the exact compression algorithm, I was trying
for a real-world "what would happen if we compress inventory.weave".

I re-ran the tests multiple times, taking the best time out of at least
10 trials (though honestly after the first or second it was very stable).

I realize that there is overhead (and even stated as much). I tried to
minimize python's overhead by creating a function and calling it
multiple times, rather than running a separate process.

On my slow machine, I found that even in-process, uncompressing was
slower than spawning zcat. On my fast windows machine, it was the other
way around, but most likely because of the process spawn overhead of
windows.

I'm not saying it is a hard and fast rule. But I think it is relevant.
Yes, I could grab a 100MB file and compress it, but that wouldn't be an
inventory.weave, which is what we are thinking about compressing.

Now what I would really like to get ahold of, is Michael Ellerman's
kernel inventory.weave, which should be a bit larger, and time how long
it would take to uncompress it, relative to the amount of time it takes
to unpack one of the weave entries.

John
=:->

> 
> Nick
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20051027/abbbd832/attachment.pgp