Python 3 Support: A Plan of Action

[Robert Collins]

On 9 September 2015 at 11:53, Richard Wilbur <richard.wilbur at gmail.com> wrote:
> On Tue, Sep 8, 2015 at 2:05 PM, Robert Collins
> <robertc at robertcollins.net> wrote:
>> FWIW here are my thoughts:
>>
>
> Thank you, Robert, for sharing your thoughts and experience.  I
> haven't any experience in porting code from Python 2 to Python 3 so
> I'm trying to learn what I can from those who have.
>
>>  - requiring a specific 2.7.x release like .10 is likely to cause
>> adoption friction
>
> I guess it was my suspicion that the port would be easier if we
> updated first to the most recent tip of Python 2.7 and then tried to
> port to some point on the Python 3.x progression.  Retaining Python
> 2.6 support certainly solves a number of issues, not the least of
> which are:
> 1.  All current users and packagers will be able to upgrade to the
> latest release of bzr without any changes to Python requirements.
> 2.  Removes some of the pressure on plugin maintenance as they will
> likely continue to work fine on older versions of Python.
> 3.  All current developers can participate in the development effort
> without the need to install a newer version of Python.
> 4.  Lowers the bar for new bzr releases (still support Python 2.6)
> while raising the ceiling (supports Python 3+).

Yup.

>>  - lots of code bases have found 2.6+ and  3.2+ to be fairly easy to
>> port to (if visually imposing due to no u'' support in 3.2).
>
> Is there a good reason to use 3.2 when it looks like 3.3+ should be
> possibly even easier?[2]  (Python 3.5 is expected to be released 13
> Sep 2015[0])

I don't think there is a compelling reason to use 3.2. My point there
was just that its been feasible since 3.2 came out (3.1 single-source
with 2.x really wasn't feasible).

>>  - 2.6+ and 3.3+ is pretty much a no-brainer: sure there are the odd
>> thing that can't be done trivially, but there's no fundamental issue
>> preventing single-code-base high performance code : and nothing I
>> recall from even the most performance sensitive gnarly bits of bzr
>> that would pose a problem.
>
> Upon reading more deeply the document I referenced when I first laid
> out some plans, "Easy,
> clean, reliable Python 2/3 compatibility"[1], I find they mention that
> the compatibility library they use, "future", also supports creating a
> codebase compatible with Python 2.6 to Python 3.3+.[2]

Oh, I missed that. FWIW I find six very very good to work with. I very
much dislike 2to3 - it was a neat idea, but IME its only suitable for
big-bang upgrades, which very few projects are willing to do.

>> So my suggestion, for what its worth, is to add a non-voting check to
>> PQM for some arbitrary version of Python 3.3+, and then start
>> iterating: there's no need IMO to remove support for any already
>> supported version of Python - it won't materially help the porting
>> process.
>
> Sounds like a fine suggestion!  Is a 'non-voting check' a way of
> getting the testsuite run under Python 3.3+ without rejecting branches
> which fail that check?  How do we get diagnostic output from that
> check?  I guess another thing that we would want is to be able to
> configure which module(s) must pass the testsuite under Python 3.3+ in
> order to accept the branch.  That way we can merge in changes to
> support Python 3 one module at a time.

Yes. So the process would look something like this:

1) Setup some mechanism for testing on (lets say) 3.4. That could be a
buildbot reading from bzr's trunk and reporting post-commit, or it
could be a patch to the Makefile target pqm uses to run the test suite
on 3.4 after running it under 2.7. It would then include the output
from that. (PQM might need a small patch to include more output on
successes).

2) Start making the code base importable - this will be fundamentally changing

except expr, y:

into

except expr as y:

(thats about the only major syntax incompat offhand).

3) Make the test suite executable - deal with any niggling
incompatibilities that may exist, such that it will try to run (and
everything may fail).

4) starting with the lowest layers start auditing the tests and
getting things working. E.g. one thing to look out for is byte vs
unicode correctness. bzr may well want to keep versioned content in
bytestrings right up until they are rendered to users - and thats fine
- but the test suite /could/ have irregularities that the implicit
decode in Python 2.x masks.

5) make pqm run both the 2.x and 3.x tests and fail if either error.

-Rob


-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud