When should Python 3.3 become the default?

[Matthias Klose]

On 19.10.2012 21:41, Steve Langasek wrote:
> Hi Barry,
> 
> On Thu, Oct 18, 2012 at 05:04:35PM -0400, Barry Warsaw wrote:
>> Continuing on discussions we've had at previous UDSes, and looking ahead
>> to the probably schedule for Python 3.4, it is almost certain that we'll
>> carry Python 3.3 as the default Python 3 for the next LTS (14.04).
>> Whether we'll also have Python 3.2 or 3.4 in 14.04 LTS is an open
>> question we don't need to decide right now.
> 
>> Clearly we're going to enable Python 3.3 as a supported Python 3 version 
>> when the 'raring' archive opens up.  The question is whether 3.3 should
>> be the default Python 3 as soon as the archive opens, or whether we
>> should keep 3.2 as the default, enable 3.3, and watch for any fallout
>> before making the switch later in the 13.04 cycle.
> 
> There seem to be a lot of big unknowns about how much work is needed, so
> I'm not sure it helps for me to weigh in, but here's my two cents anyway.
> 
> Supporting multiple versions of python3 in our source package tooling is 
> obviously important over the long term.  However, supporting multiple 
> versions of python3 is never a goal per se for an Ubuntu release (or, for 
> that matter, a Debian release): it's something we do only when we need to, 
> in order to support applications that don't manage to all be ported to the 
> same python version at the same point in time.

Delaying the use of "tooling" support until after the next LTS is too late.
It's not just the tooling, but also the use of a common dist-packages across
python versions. I'd like to see this used now for the first time with a
relatively small number of packages, instead of full blown set of packages for
the 3.3/3.4 transition.

> The set of existing software in quantal that depends on python3.2 is still 
> comparatively small: less than 500 packages all told (source+binary).  I 
> haven't sorted through to see how many source packages this actually is,
> but it's probably closer to 300 total.

$ apt-cache -n search python3-|wc -l
235

which may come down to 140 sources, and many of these are architecture
independent packages. Didn't count, but I think we come down to 10% of your
original estimate.

> If the python3.2->python3.3 porting work is small enough to do in one
> cycle, I think it's beneficial for us to focus on this porting exclusively
> and *not* worry about making packages work with both python3.2 and
> python3.3,

If you need fixing a package for a new python version, how often do you break
a previous version? Will upstreams even accept this kind of patches?

> - It avoids us blocking on bugs in packages / tooling that prevent 
> supporting multiple versions of python3 today.  We obviously need to fix 
> those *eventually*, but it's not something that's going to be a priority in
> Debian right now due to the freeze, so if we don't have to deal with it
> right now we'll spare ourselves having to carry a delta.

I think the tooling issues are overrated. Every package I checked with doesn't
show these issue (zope.interface, lxml, dbus-python).  Starting with a package
like pyqt4 known for issues and drawing conclusions for other packages doesn't
really help.

> - As we identified at last UDS, there are advantages to porting directly
> to python3.3 when coming from python2.  Getting python3.2 entirely out of 
> the picture should, over the slightly longer term, buy us back some effort
> when it comes to the python2->3 transition.

I don't see this as an argument for keeping 3.2/3.3 packages in parallel for a
while. People won't start migrating packages from 2 to 3.

If you think that the reintroduction of u"" strings will help porting, I'll
backport this to 3.2 temporarily (provided that it doesn't end up in a release).

> - It sends a clear signal to upstreams that they should be targeting 3.3, 
> not 3.2.  Yes, this may again hurt our python3 support in the short term 
> because some software supports python2 + python3.2 upstream and not 
> python3.3; but I think getting python2 off the CD for 13.04, only to have 
> to ship both python3.2 and python3.3 instead, is not really a win for us.

If an upstream already has support for python3, it won't skip a python3
version. And upstreams wanting to do development in raring can do so too, as
python libraries are already available for 3.3 (when we fix the other build
issues unrelated to the tooling).

Having python3.2 not on the CD is out of question, I'd see it rather removed
for 13.04.

> - Python 3.3 has multiarch support and 3.2 does not.  So keeping 3.2
> around has implications for cross-bootstrapping of Ubuntu, which is also on
> the list of topics for UDS.

no, that's a bogus argument. These packages build-depend on python2.x too, so
every python version needs to support multiarch for this. However here we are
still missing multiarch support in the tooling (dh_python3). and cross
building extensions is another story ...

> 
> So here's my strawman proposal.
> 
> 1) Open raring with python3.2 as default, and only, supported python3 
> version. 2) Fix the issues that prevent dh_python3 being used for
> python3.3, ASAP. 3) Stage rebuilding all of the python3 revdeps against
> python3.3 in a ppa. Identify all build failures.  Determine if these are
> tractable to sort out for raring.  If not, go to step 6. 4) Iterate in the
> PPA (pushing the necessary support patches to Debian, upstream, or raring
> as we go) until enough of the set is buildable with. python3.3 that
> uploading it is not going to be disruptive to other development.  Focus in
> particular on python3.3 compatibility in those packages which build-depend
> on python-all-dev. 5) Upload to raring, switching to python3.3 in one go. 
> 6) If we determine that we can't make it all the way up the hill to 
> 3.3-only in time for raring release, back off: make python3.3 a supported
> version, but make 3.2 the default.
> 
> Please tell me if this sounds crazy.

Point 2) is (afaics) done.  I do not want to rely on the 3.3 work being done
in a PPA only, this time is lost during the development cycle. It doesn't
expose 3.3 until it's the default, and things like the dbus-python/3.3 issue
mentioned earlier in the thread show that it's not always easy to get these
PPA configurations correct (and yes, we had some issues back with the 2.6/2.7
tests in a PPA too at the end of the last year, which then came up as a
surprise in the archive).

So lets open with 3.3 as a supported version from the start, and drop 3.2 when
3.3 is made the default (or shortly after).  Adding a new version doesn't
break the existing packages in the archive, issues are usually only exposed at
build time. I think it's a must to check for the common site/dist-packages
layout. If people do not want to address the pyqt "tooling" issues, these
should be at least documented in the bug tracker.

6) is not really an option (but maybe it's expected from your position to
provide such a fallback ;)

  Matthias