[RFC] locking decorators and iterators

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Aaron Bentley wrote:
> John Arbash Meinel wrote:
>>> Aaron Bentley wrote:
>>>
>>> Looking at the output of which entries failed after the new decorators,
>>> though, I can see that 'revision_trees' and 'get_deltas_for_revisions'
>>> are both things that should require a lock, and that they are both
>>> iterators which can actually help performance (being able to read ahead,
>>> but return things as they are available/in the requested order)
> 
> Given that repositories are append-only, do those really need locks?
> Read locks just give you caching, anyhow.
> 
> But maybe it would be enough to assert that a lock is held inside the
> iterator.
> 
> Aaron

Well, it enables caching of all the knit indexes, and you really don't
want to be re-reading them for every operation. Now, it is actually just
caching the KnitVersionedFile object, so it may be as long as you are
re-using the same reference, you won't need to re-read.

But I don't think the code has been audited for that sort of thing. What
I could see is if you ask for multiple revision trees, the deep call may
start to re-read the inventory.kndx file.

I haven't tracked it through thoroughly, but 'revision_tree' calls down
into get_revision_inventory, which would have to read the inventory.kndx.

But it looks like get_deltas_for_revisions() only calls
self.revision_tree() to get the 'null:' tree. So we should be safe. 2
calls to get_deltas_for_revision() or revision_trees() will have to
re-read everything, but within a single call it doesn't have to be locked.

It takes a bit to audit code for that sort of thing, though. Versus
being able to say "all code from this point on is executed with caching
enabled". Which is what the lock decorators do.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFF0dsnJdeBCYSNAAMRAhW2AKC9dq0Y6ixZjuKLrfFC78Y+gtcVMACfWShC
xrrZH8JDgfNQhEhUGCk9meM=
=+K9n
-----END PGP SIGNATURE-----