Getting ready for Python 3.5

[Barry Warsaw]

[Apologies for the cross-posting! -BAW]

For Ubuntu 15.10 (Wily Werewolf), we want to make Python 3.5 the default
Python 3 version.  It's currently undecided whether we will keep Python 3.4 as
a supported version, but a lot of that depends on how easily an archive port
to Python 3.5 goes.  Ideally, we'd be able to make the switch to 3.5 now ahead
of the planned 16.04 LTS release.

As part of this work, we've done a partial test rebuild of packages in the
archive that depend in some way on Python 3.  For now, this is an x86 only
partial rebuild in a PPA.  In this PPA, we have Python 3.5 as the default
Python 3 version, with 3.4 still enabled.

You can see the results so far here:

https://launchpad.net/~pythoneers/+archive/ubuntu/py35asdefault/+packages

TL;DR: of 1065 uploads, we have ~64% success rate.  Some caveats:

 * These are i386 and amd64 only, so we're still in the dark about other
   architectures.

 * These are build tests only.  While many builds do run the package's test
   suite, not all packages have test suites, or not all are enabled.  Also, we
   are not running what are called DEP-8 tests, which are various additional
   smoketests that are run after the package is built, on the built package
   (e.g. install the package and all its dependencies in a chroot and see if
   it can be imported in both Python 2 and 3).

 * Some failures are due to dependency build order, so a simple rebuild may
   succeed.

 * Some failures may not be new.  Because a lot of packages don't get new
   uploads for every new Ubuntu version, they may be failing to build for
   reasons unrelated to Python 3.5.

 * We may have missed some packages which declare their build dependencies in
   a way that got past our rather simplistic filter.

Our plan is get the success rate up on the PPA, filing and fixing bugs
upstream where possible, then to set up a full archive test rebuild, again
with 3.5 as default and 3.4 as enabled, to see what other failures may occur.
This full archive rebuild will include all the other architectures, and it's
possible packages will build on x86 but fail on some other platform.  We're
also planning on setting up a QA stack to run the DEP-8 tests for packages
that have them.

In the meantime, you can help!

I've started a wiki page listing the backward compatibility breaks I've found
so far.  Feel free to add to this, or look into the linked issues.  Contribute
to the wiki or the linked issues.

https://wiki.python.org/moin/PortingToPy3k/34to35

Build and install Python 3.5 from source and run your package's test suite in
a virtual environment or in tox.  (tox 2.1 supports Python 3.5 but it isn't
yet available in Debian/Ubuntu - no worries, install tox in a virtualenv and
use *that* to run your test suite.)

Create a chroot with the py35asdefault PPA enabled, and do some porting there.
Here's a good guideline on how to use a PPA.

https://help.launchpad.net/Packaging/PPA
https://launchpad.net/~pythoneers/+archive/ubuntu/py35asdefault

Examine the build failures and see if you can identify or fix the problem.
Start with the package details page

https://launchpad.net/~pythoneers/+archive/ubuntu/py35asdefault/+packages

and drill down into the console logs for specific failures by clicking on the
'amd64' or 'i386' links next to any big red X you see, then clicking on the
'buildlog' link.  E.g.

http://tinyurl.com/pnpjtv6

Scroll down near the bottom, which is where the failure will most likely be
evident.

Release new versions of your packages with Python 3.5 support to PyPI so the
Debian maintainers and Ubuntu developers can begin to upload compatible
versions.  If you're a developer for other Linux distros or platforms, let's
work together!  (As is often the case, we'll trail blaze on Ubuntu and push
the results upstream to Debian as much as possible.)

Follow up here on any of the CC'd mailing lists, email me directly, or ping me
on IRC (nick 'barry' on python-dev, ubuntu-release @ freenode, debian-python @
OFTC).

Python 3.5 is in beta, with a final release scheduled for September 2015 (see
PEP 478).  There's still plenty of time to fix or adjust to issues we find but
there are a ton of packages, so all help is greatly appreciated.

Let's make Python 3.5 the easiest upgrade ever.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20150623/48c9e66d/attachment.pgp>