Is it worth back porting PEP 3147 to Python < 3.2?

[Piotr Ożarowski]

[Barry Warsaw, 2010-04-22]
> On Apr 21, 2010, at 12:01 AM, Piotr Ożarowski wrote:
> >2to3 is not reliable, at least not for now.
> 
> I agree that there's no way we can just enable it by default.  Too many
> upstream packages need modifications to work in Python 3.  However, for those
> that are Python 3-ready, it's a useful tool.  The question then is whether we
> want to enable it for those packages that it works with.

I was referring to the idea of using 2to3 at runtime. If 2to3 passes
tests at build time and upstream author/maintainer is satisfied with
results, we should use it, sure.

> How much of the transition testing is automated?

last time I was testing my cdbs / python-central changes, I followed
these steps:

* create cowbuilder env. with custom packages
* download all packages that are affected by the change
  (build depend on python-central / python-support /
  python{-all,}{,-dev}, etc. - grep-dctrl is very handy here)
* rebuilt all of them using slightly modified[1]
  /usr/share/doc/pbuilder/examples/rebuild/buildall script
* read debc output, check if everything is ok, if not, find a pattern
  and grep for it in other files, then use mass-bug[2] to file bugs
* read FTBFS logs, and do the same as with debc output files

[1] f.e. save `debc package.changes` output. It takes few days to
    rebuilt all packages so make sure the script works correctly and
    you have enough disk space :)
[2] I actually used my own Python script (/me maintains too many Python
    template engines to not use them whenever he can ;)

> It would be very interesting
> for example, to have a test framework that could run any combination of Python
> packages against various versions of Python, and get a report on the success
> or failure of it.

IMHO we should encourage maintainers to enable tests and
build/bytecompile .py files for all supported Python versions, even in
Architecture: all packages (I found few bugs in my packages this way).
You can test if binary packages pass the byte compilation, but unit
tests are available usually only in source packages.

> > /usr/share/python3/foo/__init__.py
> > /usr/share/python3/foo/bar.py
> > /usr/share/python3/foo/3.2-/bar.py
> > /usr/share/python3/3.1-3.5/baz.py

this should be /usr/lib/python3/... actually

> Those are the symlinks, right?  E.g. /usr/share/python3/foo/3.2-/bar.py would
> symlink to /usr/shared/pyshared/bar.py?

no, /usr/lib/python3/foo/3.2-/bar.py is the location in .deb and the
final location of foo.py on the disk (i.e. the final runtime location -
/usr/share/python3/ will be in all pytho3.X's sys.path), that's what
PEP3147 is about, no? No more symlinks. In above example bar.py has
different source for Python 3.1 and >= 3.2. It should actually be:

 /usr/lib/python3/foo/bar.py
 /usr/lib/python3/foo/3.1/bar.py

or f.e.

 /usr/lib/python3/foo/bar.py
 /usr/lib/python3/foo/-3.3/bar.py

if bar.py is the same in 3.1&3.2 and a little bit modified in >= 3.4

> I like the idea of shipping symlinks

symlinks will be shipped only in 2.X packages (if one will decide to
use new dh_python instead of pycentral/pysupport) and only if PEP3147
will not be backported (this approach requires rebuilding arch:all
packages which doesn't have many fans)

> , but I thought they were created at
> runtime to handle the case where a new version of Python was installed after
> the package was installed.  Is that the case, and if so, wouldn't all the
> packages have to be updated when a new version of Python is supported?

pycompile (invoked[3] when new version is installed/removed) will take care
of creating appropriate __pycache__/bar.cpython-*.pyc files and pyclean
will remove them (including __pycache__ directory, if not needed anymore)

[3] see /usr/share/python/runtime.d/* for examples
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20100422/d2725111/attachment.pgp