[RFC] Extract unannotated texts from Knits

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> On Wed, 2007-07-25 at 14:29 -0500, John Arbash Meinel wrote:
>> Profiling seems to put file.close() as 1s (which is the time it is
>> writing out
>> data). Which is extra strange since cp -ar is able to do it in 300ms.
>> I at
>> least wonder if cp is able to do things with memmap files, rather than
>> direct
>> access. 
> 
> What OS is this on? On linux at least, closing the file handle does not
> guarantee data flushing. (man 1 close says:
> """A successful close does not guarantee that the data has been
> successfully saved to disk, as the kernel defers writes. It is not
> common for a filesystem to flush the buffers when the stream is closed.
>        If you need to be sure that the data is physically stored use
> fsync(2).  (It will depend on the disk hardware at this point.)
> """
> 

Sure, I wasn't meaning to say it was an actual fsync to disk. Just the final
buffer flush.

> If you are seeing a slow close it may be that the file handle is
> buffered at the stdio layer, and its being flushed from there. There are
> what 650 files being written? So thats 1023/650=1.2ms per close call
> which seems far to small to represent actual disk seek and write times.
> 
> stracing the checkout may give more insight.
> 
> Rob
> 

Yeah, the buffering issues still are worrisome, since it is taking 3x as long
to just write the output. So even if we had no other overhead, it would still
take 3x as long for us to write the output files. Then again, if we could do
'bzr checkout' in 1s, I don't think I would complain.

It could still be some locality of reference in the output, but I believe Aaron
has done lots of good work in putting files in their place, so you only need to
rename top-level items. Which is why I wondered if cp was able to do memmap or
some other trick for better performance.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGqLJGJdeBCYSNAAMRAjkcAJ9mtdHz3j0c4/fI23TDsD/JTHw8BQCePz/o
hLB42L2w6nNcYSxd1ziChCU=
=VqET
-----END PGP SIGNATURE-----