brisbane:CHKMap.iteritems() tweaks

[Robert Collins]

On Tue, 2009-03-24 at 22:35 -0500, John Arbash Meinel wrote:
> 
> So there is a significant amount of overhead in
> 'Access.get_raw_records', since we are reading 720*XMiB of compressed
> data, even though we only care about the first few bytes out of each
> block. And then there is a certain amount of overhead decompressing
> the
> extra 32KiB from each block (this seems to be trivial, as if I set the
> extra decompress size down to 0, it only seems to save 50ms)

Why are we reading more data than needed? (I thought I had code
[disabled] to read only the part of the compressed group needed.

> We can also look at combining 'pack recent separately' along with
> 'split chk by prefix'. But if the new overhead is that we now access
> 720 blocks on disk, rather than 6 (don't forget the extra seek? to
> read the pack header again, since this is a fresh open()), that
> doesn't improve with 'recent'.

Well, thats where mmap may come is useful for local performance.

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20090325/0329cb57/attachment.pgp