FreeBSD Ports statistics

[John Arbash Meinel]

Lele Gaifax wrote:
> John Arbash Meinel wrote:
>> 2) Tailor has a memory consumption bug. When I first run tailor, it does
>> a full 'cvs rlog' dump, and I assume parses that into changesets. It
>> then starts operating, and consumed 1GB of RAM.
>> After stopping and restarting it, it is now only consuming 120MB of RAM.
>> My best guess is that it parses the rlog contents, and then leaves the
>> Unicode string in RAM. But doesn't write that to the state file, because
>> it doesn't need it anymore.
> 
> Uhm, this clearly shouldn't happen, I'll try to figure out why it's
> keeping the usage so high.

Well, as I dig deeper, it might have to do with the state file.
It seems if you have a state on disk, it only needs to load 1 entry to
process at a time. Versus having everything loaded. This may not be it,
but it is definitely something that I saw.

> 
>> 3) Tailor is fairly slow, but not terribly so. It has taken about 4hrs
>> to create the first 2K revisions. (With approx 6000 file-level changes).
>>
>> At this point, I'm pretty sure that all of this is 'cvs' time. Tailor
>> spawns 'cvs update -d -r 1.2 filename' for *every* file content change.
>> It isn't able to combine multiple updates. And since RCS uses back
>> patches, to checkout an early version requires applying all backwards
>> patches (Thus is O(n revisions)).
>>
> 
> Maybe you already know this:
> http://progetti.arstecnica.it/tailor/wiki/CvsRepository#WhyistailorsoslowdealingwithCVS
> 

I hadn't specifically found that link, though I have figured out the
cause already.

Oddly enough, googling for darcs cvs one second delay, and some other
terms never turned up that link.

> 
> But what do you mean with "it isn't able to combine multiple updates"?
> 
> thank you,
> bye, lele.

Often (especially with the initial add, and files that tend to be
modified together) you will see something like:

cvs update -d -r 1.1 file/one

cvs update -d -r 1.1 file/two

cvs update -d -r 1.1 file/three

Technically, all of those have the same revision, and you could instead do:


cvs update -d -r 1.1 file/one file/two file/three

I realize you have to be careful, because there is no dependency that
the files are in sync, and over a long enough timeline, files tend to
get out of sync.

But especially on something like the ports tree, you have a *lot* of
files being created at the same time, and then later on being updated at
the same time.

It would be a small optimization, but it would help quite a bit when not
using a modified cvs.

John
=:->


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060901/bbf12b9e/attachment.pgp