FreeBSD Ports statistics

[John Arbash Meinel]

Lele Gaifax wrote:
> John Arbash Meinel wrote:
>> Well, I can't say exactly what is being held. My best guess was that it
>> was the rlog. All I can say is that when bootstrapping, memory went to
>>> 1GB. And then when running later, it dropped to 120MB. So it seemed
>> like rlog would be the culprit.
> 
> Yes, it may very well be: the parser gets fed basically by
> 
>   reader = codecs.getreader(encoding)
>   log = reader(output of rlor)
>   return iterator_over_changeset_from_rlog(log)
> 
> so maybe the reader is retaining its input and should be explicitly
> closed, or something like that...

Well when using an iterator like that, it could easily be holding open a
file object. I don't really know myself. I assume you are doing this
because you think it is better for not having to read in the entire file?

Is 'output' a string object, or a file? Because multiple calls to
string.encode() is usually very inefficient. It is usually better to do:

txt = file.read().decode(encoding)
rather than:
for line in file:
  txt = line.decode(encoding)

I saw some large advantages in bzr in doing the first form (versus
calling decode() about 100K times).

> 
>> Is this patch in the main distribution (or will be)? I can certainly
>> download it locally, and see if it works for me. Though I'm not sure how
>> to pull a new changeset out of 'trac' to get it into a darcs branch.
> 
> From within the darcs branch:
> 
> $ darcs pull http://darcs.arstecnica.it/tailor
> 
> you can omit the explicit address if that's where you "darcs get"ed the
> branch
> 

Yep, it worked, and I can see that it does update multiple files at once
now. So now 'cvs update' isn't as big of a deal, just 'bzr commit' time,
and some other time that I don't know what Tailor is doing. Right after
it prints the log message, it spends a couple seconds just sitting there
before it starts spawning 'cvs update'.

I'm currently at revno 14500 and 18000 files.

>>
>> And to be honest, at this point, the 'bzr commit' time is starting to
>> overwhelm the 'cvs update time'. Though every second I can shave off per
>> revision will help. (I'm up to revno 13963, and I have 17584
>> files/directories being versioned, and each commit is taking around 8s)
>> A committed inventory is now 4.1MB.
>>
>> If you can tell me how to get your patch, I would appreciate it.
> 
> Let me know,
> bye, lele.

Things seem to be going pretty slowly, but they are moving forward.
Considering there are several hundred thousand revisions to go, I'm not
terribly optimistic about it finishing soon :)

John
=:->

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060903/e7c60f25/attachment.pgp