Python 3 Support: A Plan of Action

[Richard Wilbur]

On Sun, Sep 6, 2015 at 12:48 PM, Jelmer Vernooij <jelmer at samba.org> wrote:
> On Sun, Sep 06, 2015 at 10:40:06AM -0600, Richard Wilbur wrote:
[...]
> If it was my call, I'd probably just focus on Python 2.7.10 and
> Python3, and only backport essential bug fixes to Bazaar 2.6. The port
> will take up some time as well - months, I imagine - and then there is
> more time before a release actually happens. There are (currently)
> also no must-have features in the new Bazaar that users on older
> platforms would be deprived of.

I agree, Jelmer.  Since my original plan was to release at least once
with bug fixes and features from the trunk baseline as 2.6.x, and
maybe once more with cleanup like Vincent's pep8 improvements
(2.6.x++), before we branch for bzr 2.7, there will be a little time
before we even have an opportunity to backport anything to bzr 2.6.
Right now the question is more of whether we should backport anything
from bzr 2.6 to 2.5 or 2.4.

The port itself will, I am sure, take several months during which time
there may be the opportunity to backport a fix or two.  That is a
fairly small price to pay for the progress to a Python 3-compatible
codebase.

>> 1.  Break with the past at the bzr v2.6 versus v2.7 boundary and
>> backport fixes to 2.6 for some acceptable length of time to allow
>> distributions and their users to catch up with the python version
>> requirements imposed by bzr 2.7.
> This seems like quite a lot of effort to go through, especially
> considering the number of people working on Bazaar at the moment
> The low change rate does mean there are not a lot of changes
> to backport so it might not be a problem at the moment. If this picks
> up, it might not be tenable.

I don't expect the rate of fixes or new features to appreciably rise
during the port effort unless the number of contributing developers
rises significantly (possibly inspired by all the visible activity?).
Thus I expect to actually have fewer changes to backport owing to the
contributing developers concentrating on the port effort.

>> 2.  Create or use some compatibility implementation of new python
>> features that we use in bzr 2.7+ with a sundown date (planned
>> deprecation) an acceptable distance in the future "to allow
>> distributions and their users to catch up with the python" versions
>> that first implemented those features in the language.
>>     In other words, determine the lineage of new language features
>> we'd like to use (what version of python introduced them and when was
>> it released), then if it has not been available long enough to satisfy
>> our criteria for general availability/wide adoption, create or use a
>> feature implementation compatible with python versions that satisfy
>> said criteria.
> We've been doing this in Bazaar for a long time; there are various
> places where we fall back to replacement code if the version of Python
> in use doesn't provide certain utility functions.
>
> This works well for e.g. missing library functions. It's harder to
> provide compatibility for some other features, e.g. bytes/unicode
> behaviour - you essentially end up inventing a third language that is
> compatible with both Python2 and Python3 and you need to be
> disciplined about what features you use so that they work in both the
> old (2.6) and new world.

Good to know there is experience and expertise in this tactic in the
bzr community.

I see that I wasn't very clear in stating that I was thinking of
creating or using compatibility implementations to bridge between
python 2.7.latest (currently 10) and 2.7.(general availability/wide
adoption).  By the same token, between python 3.latest (soon 5) and
3.(general availability/wide adoption).  I was not even entertaining
the idea of trying to support python 2.6 in the new bzr codebase
(v2.7).  (My old world was python 2.7.(general availability/wide
adoption) and python 3.(general availability/wide adoption), new world
python 2.7.10 and python 3.5+.)

Point taken about exercising care in creating a language compatible
with both Python2 and Python3--that could get out of hand.

In light of the comments, then it seems that maybe a hybrid approach
would be best:
1.  bzr 2.6.x will continue to only require python 2.6 and receive
backports of important fixes and features which improve
interoperability.
2.  Create a good method of determining which versions of python 2.7
and 3 satisfy our criteria for general availability and wide adoption.
3.  bzr 2.7 will require a suitable version of python 2.7 or 3
determined using the method from #2 above (old world) but should work
in the latest releases (new world) of python 2.7.10 and 3.5(soon).
Determine the feasibility of bridging the feature gaps between the
minimum required versions of python and the latest releases by using
or creating compatibility implementations.  I would encourage
developers to work and test in a variety of python releases spanning
the range from the minimum required to the latest releases.

[...]
>
>> Another possibility would be to establish a consensus on what we
>> consider a representative sample of distributions in current use and
>> consider which versions of python they ship.  Then only change
>> expected python versions when those distributions update their
>> offerings and are in wide adoption.
> This seems like a reasonable approach.

So let's figure out what we will be looking at to determine our
representative sample--we have the advantage that this is a fairly
slow-moving target and we have computers at our disposal should the
sample size exceed what we wish to deal with manually.

Taking nominations for Gnu/Linux distributions and other operating
systems we should consider when determining "general availability and
wide adoption" as regards python releases.  (These would also be
likely candidates for maintaining bzr OS support, testing, and
packaging/installation instructions/installers as they imply installed
users.)

Ubuntu (which releases are important?)
Debian (which releases are important?)
?

Thank you all, as always, for participating in the discussion.
Richard