[Bug 770566] Re: Intermittent crashes copying files involving symbolic links

[Matthias Klose]

fixed in python 2.7.3 in precise

** Changed in: python2.7 (Ubuntu)
       Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python2.7 in Ubuntu.
https://bugs.launchpad.net/bugs/770566

Title:
  Intermittent crashes copying files involving symbolic links

Status in Python:
  Fix Released
Status in Enhancements to Python's distutils:
  Fix Released
Status in “python-distutils-extra” package in Ubuntu:
  Fix Released
Status in “python2.7” package in Ubuntu:
  Fix Released

Bug description:
  The approach used by python-distutils-extra to install icon (and
  perhaps other) symbolic links can proceed or crash depending upon the
  random order in which icon file copies are executed.

  The crash happens while trying to create the debian/tmp/ file tree
  when apport's make processess calls python-distutils-extra through
  setup.py using the following command:

    python setup.py install \
    --root="/home/gruber/packages/uapport/debian/tmp/" \
    --install-purelib=/usr/lib/python2.7/site-packages/ \
    --no-compile --install-layout=deb

  For this case I received the following message from pbuilder while trying to build the apport package for ubuntu:
  copying data/icons/scalable/mimetypes/text-x-apport.svg -> /tmp/buildd/apport-1.20.1/debian/tmp/usr/share/icons/hicolor/scalable/mimetypes
  error: /tmp/buildd/apport-1.20.1/debian/tmp/usr/share/icons/hicolor/scalable/mimetypes/text-x-apport.svg: No such file or directory
  make: *** [install/apport] Error 1
  dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
  -----------------
  There are icon files in two different directories, one a symbolic link to the other.

  In particular the source icon files are:

  -rw-r--r-- 1 gruber gruber 16628 2011-04-25 12:52:10.493242000 -0400 data/icons/scalable/apps/apport.svg
  lrwxrwxrwx 1 gruber gruber 18 2011-04-25 12:52:10.409823944 -0400 data/icons/scalable/mimetypes/text-x-apport.svg -> ../apps/apport.svg

  these are to be copied by python-distutils to:
  lrwxrwxrwx 1 gruber gruber 18 2011-04-25 17:44:10.579475687 -0400 debian/tmp/usr/share/icons/hicolor/scalable/mimetypes/text-x-apport.svg -> ../apps/apport.svg   (1) and
  lrwxrwxrwx 1 gruber gruber 18 2011-04-25 17:44:10.579475687 -0400 debian/tmp/usr/share/icons/hicolor/scalable/apps/apport.svg    (2)

  python-distutils-extra makes the symbolic link itself at the very end
  of auto.py, making destination directory for the symbolic link (1) in
  the process. python-distutils-extra command/build_icons.py add both
  source and destination files to distribution.data_files.

  python-distutils then has the responsibility to finish the work on
  these icons. If the above entries were added by command/build_icons.py
  with the entry for the target icon file first, the build process
  completes. When file_util.py gets to the data_file entry for the
  target it builds its directory and then copies apport.svg to it. Next
  it gets to the entry for the mimetypes directory and checks to see if
  it is up-to-date and one way or the other proceeds.*

  If the two data_file entries are produced in the opposite order
  file_util.py checks for the existence of destination directory (1) and
  then tries to find out whether the file is up-to-date. It isn't aware
  of the symbolic link and doesn't check for the existence of
  destination directory (2), which doesn't yet exist, so the newer
  function call fails when the attempt to stat the destination file is
  made.  While python-distutils has functions for producing hard and
  symbolic links that code is not being accessed for this operation.

  On my system the mimetypes directory has always come first and the
  build always fails. On launchpad the apps directory (target of the
  symbolic link) is usually listed first and normally the build proceeds
  without event.

  The order of the entries is determined by glob.glob() calls in
  command/build_icons.py. The documentation for glob.glob()
  (http://docs.python.org/library/glob.html) says the python library
  uses the os.listdir() function to produce its results, and
  http://docs.python.org/library/os.html says that "The list is in
  arbitrary order." so whether the setup.py command and the build
  completes or not is also arbitrary.

  * There is a second difference in execution between pbuilder and
  launchpad's build farm. The build farm keeps file modification time in
  seconds and python-distutils' dep_util.py uses subsecond floating
  point stat.st_mtime values around line 85 while copy times are
  preserved in file_util.py using stat[ST_MTIME], an integer value. This
  doesn't matter on an ext3 file system but causes the destination file
  to look older than the original on an ext4 file system, which tracks
  subsecond times (note the file times above). The symbolic-link-unaware
  copy process in this case erases the symbolic link from destination
  directory (1) and replaces it with the actual source file (replaces
  the symbolic link with the actual file). Using the comparison at the
  precision of 1 second is the better way to solve this to avoid
  floating point precision problems from continuing to confuse the
  "newer" function in dep_util.py.

To manage notifications about this bug go to:
https://bugs.launchpad.net/python/+bug/770566/+subscriptions