[MERGE] import both c-extension and python module with the same name for testing

[Alexander Belchenko]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Arbash Meinel пишет:
> Alexander Belchenko wrote:
>> Lukáa Lalinský ?8H5B:
>>> On `t, 2007-08-09 at 10:55 +1000, Martin Pool wrote:
>>>> To try and summarize the thread: there are a few choices
>>>>
>>>>  1- merge something like Alexander's patch; then any module that wants
>>>> a doubly-implemented module can just simply import it and will get
>>>> either the C or python version
>>>>  2- put the try/import/except/import into things that need to import it
>>>>  3- have a pure-python wrapper module that imports whichever
>>>> implementation is available
>>>>
>>> [...]
>>>> Anything else?
>>> Bzrlib already has two Pyrex extensions, but it is likely that there
>>> will be more of them in the future. What about compiling them all into a
>>> single extension (e.g. bzrlib._ext). The modules contain quite low-level
>>> code which needs to be imported in almost all cases, anyway. I'm not
>>> sure how big overhead have dynamic linkers on all bzr-supported
>>> platforms, but I expect it to be bigger than for example the overhead of
>>> try/except in case of importing the Python modules.
>> It will lead to mess in namespace of this big-one extension module.
>> I don't think it's a right thing.
> 
> Pyrex does have support for compiling multiple modules into a single
> file. I'm just not sure how it exactly works.

Pyrex translate each .pyx file into one standalone C-extension module.
So, compiling several pyx files in one big module will not work as you want.
Instead you have several small modules in one shared library.
Unfortunately, Python used name of shared library to looking for
name of entry point (or initialization function) for C-extension.
As you can see from typical C-sources there is only one public
name exported from module: initXXX. This function initialize module
namespace. All C-function inside C-extension module declared as
static and only available via special namespace table.

So compiling 2 pyx file in one Extension do wrong thing:
you have 2 public names: initm1 and initm2.
But `import m1` give access only to m1. To import m2 you need
rename shared library to m2.[so/pyd] (may be instead of rename one could
use symlink, I don't know and don't care because it's not win32-safe).

Attached simple test that show this.

> Specifically, you can do something like:
> 
> Extension('bzrlib._ext', ['module_one', 'module_two'])

AFAIK, this used to compile pure C-sources along with pyx,
i.e.

Extension('module_pyx', ['module_pyx', 'source.c'])

Pyrex allows textual include of content into pyx module,
as described in Language overview document:

- -------------------------------------------------------------
The include statement
For convenience, a large Pyrex module can be split up into a number of files which are put together
using the include statement, for example
include "spamstuff.pxi"
The contents of the named file are textually included at that point. The included file can contain
any complete top-level Pyrex statements, including other include statements. The include statement
itself can only appear at the top level of a file.
The include statement can also be used in conjunction with public declarations to make C functions
and variables defined in one Pyrex module accessible to another. However, note that some of these
uses have been superseded by the facilities described in Sharing Declarations Between Pyrex Modules,
and it is expected that use of the include statement for this purpose will be phased out altogether
in future versions.
- -------------------------------------------------------------

So, using

include "spamstuff.pxi"

will lead to namespace pollution.

> So far, I've only done:
> 
> Extension('bzrilb._foo', ['bzrlib/_foo.pyx'])
> 
> I have actually thought it could be good to have a single extension. My
> big reason is that I think loading 1 extension will be faster than
> loading multiple.
> 
> What I don't understand is how you are able to extract code when you do
> that sort of thing. If I have _foo.pyx with a function 'foo'. And I
> compile it to "bzrlib._ext", do I "from bzrlib._ext import foo" ?
> 
> If that is the case, then Alexander has a good reason to be worried
> about namespace polution. We certainly expect that each file will have
> its own namespace.
> 
> Anyway, if someone wants to take the time to try different pyrex
> situations, I would be interested to hear what happens.
> 
> John
> =:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGvD8NzYr338mxwCURAgDNAJ9Rq9NV22PWAZnCRsimimL08W4gzwCfVk3J
y7r6deJIGw+XGHXYzLqMcl4=
=xdQs
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: setup.py
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20070810/17e4e02d/attachment.diff 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: m2.pyx
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20070810/17e4e02d/attachment-0001.diff 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: m1.pyx
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20070810/17e4e02d/attachment-0002.diff