[RFC/MERGE] Updates to decorators

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Bennetts wrote:
> John Arbash Meinel wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> I have 2 updates to our current decorators, one follows on the other
>> one, but is a little more controversial.
>>
>> Basically, I've been doing lsprof testing, and I've been getting
>> frustrated with the 'read_locked()' functions. Because in a big test,
>> you can have 10 or more @needs_read_lock functions. So when you are
>> tracking up from the bottom, or down from the top, you frequently hit a
>> barrier which makes it difficult to tell who is calling what. (10
>> functions call "read_locked()" and read_locked calls 10 other functions,
>> so which is being called without going to the source?)
>>
>> The first pass changes the needs_read_lock and needs_write_lock
>> decorators so that they use 'exec' to create a function with the a
>> mostly correct name (plus a _read_locked suffix). In testing, it was
>> enough to make sure that functions could be uniquely determined. In
>> theory if we have Branch.get() and Repository.get() they would end up
>> having the same function signatures.
> 
> Why isn't the existing setting of __name__ adequate to do this?  Maybe __module__
> needs to be set as well?

No, because lsprof seems to be reading the name from somewhere else.
Even though we set __name__ (which is sufficient for help(foo) to print
'foo') lsprof still reports that we are calling the function named
"read_locked". And because it has the same signature (it is a function
whose calling name, filename, and firstlineno are the same) all of them
get merged together.

If we could figure out where 'lsprof' gets the function names from, and
then set *that* to the correct name, it would work.

My guess is that lsprof uses a different handle because it is going
through the profiling hooks, rather than introspecting the functions.

Looking at the profile.py module, I can trace down until I get to:

fcode = frame.f_code
fn = (fcode.co_filename, fcode.co_firstlineno, fcode.co_name)

So specifically it seems to be using:
function.func_code.co_name

rather than function.__name__

And unfortunately co_name is a readonly attribute (as are pretty much
all of the members of function.func_code).

So the only way I've found to make it work would be to either change the
profiling module (and I'm not sure how to do that, since it has a Frame
and not a real Function object), or to compile a custom function. I've
chosen the latter for now, I would welcome any way to do just override
the internal name, though.

> 
> [...]
>> The second patch extends the first one by making the wrapping created
>> function also include the same parameters as the original function. It
>> uses 'inspect' to do most of the introspection. What this really changes
>> is the results when doing help(function). Specifically the difference is:
>>
> [...]
>> fetch(self, *args, **kwargs) unbound bzrlib.branch.Branch method
> [...]
>> fetch(self, from_branch, last_revision=None, pb=None) unbound
> 
> This is a limitation of the pydoc and epydoc tools, but PyDoctor at least
> generates the right output with the existing decorator.
> 
> -Andrew.
> 
> 

Is it because it does a static analysis rather than import and getdoc(foo)?

As an aside, does it also get the documentation right for deprecated
functions? Since our decorator adds the information that the function is
deprecated.

Anyway, my guess is that PyDoctor trades one form of incompleteness for
another, though because of the frequency of occurrence, PyDocter
probably gets it correct more often.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFp65nJdeBCYSNAAMRAnZ7AJ9tYOHaiswjKnx0b/Yy0ZJYB2ZqMwCeKF7t
y4EkNLXCgULBXsQLgjs/hlA=
=lxHT
-----END PGP SIGNATURE-----