[Bug 2002043] [NEW] Python extension modules get built using wrong compiler flags with python2

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Ubuntu Foundations Team Bug Bot (crichton):

Compiling a Python extension using Python2 (Python 2.7.18) is making use
of wrong compiler flags, hence dropping required optimizations when
required. This is happening only when python2 is installed from Ubuntu's
repositories. By default, Python's distutils module uses compiler and
linker flags used to compile Python itself to be used to compile
extensions.

Steps to reproduce:
1) On Ubuntu 20.04, install python2 using apt package manager.
2) After successful installation, verify the CFLAGS variable from sysconfig module. On my machine, the output is 

Python 2.7.18 (default, Jul  1 2022, 12:27:04)
[GCC 9.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS')
'-fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=. -fstack-protector-strong -Wformat -Werror=format-security  '

3) Build a test extension module using python2 and verify the compilation flags. 
python2 setup.py build_ext --inplace

Output from below command is not matching with our expected above CFLAGS. 
aarch64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-vvQ8AI/python2.7-2.7.18=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c testmodule.c -o build/temp.linux-aarch64-2.7/testmodule.o


On further investigation, it looks like Ubuntu's specific patch applied on libpython2.7-stdlib package is altering the original upstream implementation of distutils/sysconfig.py code.

Package - https://packages.ubuntu.com/focal/libpython2.7-stdlib
Patch - http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz

Below is the code block which is causing the issue, where the presence of configure_cflags is modifying cflags. This code is result of ubuntu's patch and doesn't come directly from upstream python implementation.
File - /usr/lib/python2.7/distutils/sysconfig.py
Part of code block:
        elif configure_cflags:
            cflags = ' '.join(str(x) for x in (basecflags, configure_cflags, extra_cflags) if x)
            ldshared = ldshared + ' ' + configure_cflags


I don't see problem on Python3 though we have extra code added from patch there as well. Patch used on python3, is not modifying the cflags completely and instead appending new flags to cflags.
On python3 (tested on Ubuntu 20.04)
File - /usr/lib/python3.8/distutils/sysconfig.py
Part of code block which doesn't alter cflags completely
        elif configure_cflags:
            cflags = cflags + ' ' + configure_cflags
            ldshared = ldshared + ' ' + configure_cflags


Request to update the python2 patch to behave similar to what is been done on python3.

** Affects: python2.7 (Ubuntu)
     Importance: Undecided
     Assignee: Ghadi Rahme (ghadi-rahme)
         Status: In Progress

** Affects: python2.7 (Ubuntu Bionic)
     Importance: High
     Assignee: Ghadi Rahme (ghadi-rahme)
         Status: In Progress

** Affects: python2.7 (Ubuntu Focal)
     Importance: High
     Assignee: Ghadi Rahme (ghadi-rahme)
         Status: In Progress

** Affects: python2.7 (Ubuntu Jammy)
     Importance: High
     Assignee: Ghadi Rahme (ghadi-rahme)
         Status: In Progress

** Affects: python2.7 (Ubuntu Kinetic)
     Importance: High
     Assignee: Ghadi Rahme (ghadi-rahme)
         Status: In Progress

** Affects: python2.7 (Ubuntu Lunar)
     Importance: Undecided
     Assignee: Ghadi Rahme (ghadi-rahme)
         Status: Won't Fix

** Affects: python2.7 (Ubuntu Mantic)
     Importance: Undecided
     Assignee: Ghadi Rahme (ghadi-rahme)
         Status: Won't Fix


** Tags: patch se-sponsor-halves
-- 
Python extension modules get built using wrong compiler flags with python2
https://bugs.launchpad.net/bugs/2002043
You received this bug notification because you are a member of Ubuntu Sponsors, which is subscribed to the bug report.