[Bug 1599459] Re: cannot bootstrap from repositories without InRelease files when --debian-installer set

[Jacob Doherty]

I believe we may be striking this bug using crouton and running Ubuntu in a chroot.
See: https://github.com/dnschneid/crouton/issues/2966

For us it's likely upstream as crouton uses the latest debootstrap
release (not the ubuntu package) but guessing both are affected and
hoping the ubuntu package maintainers might be able to verify and raise
with their upstream.

The symptoms users are reporting are that the InRelease file does not
exist for precise/trusty, it will re-try several times and then stop -
seemingly not checking for a Release file.


** Bug watch added: github.com/dnschneid/crouton/issues #2966
   https://github.com/dnschneid/crouton/issues/2966

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to debootstrap in Ubuntu.
https://bugs.launchpad.net/bugs/1599459

Title:
  cannot bootstrap from repositories without InRelease files when
  --debian-installer set

Status in debootstrap package in Ubuntu:
  Confirmed
Status in debootstrap package in Debian:
  New

Bug description:
  Version: 1.0.78+nmu1ubuntu1

  I am using debootstrap in my own minimal system bootstrapper, and am
  making use of the progress information reported to FH 3 as enabled by
  the --debian-installer command-line flag.

  Unfortunately, using this flag causes the behaviour of debootstrap to
  change, specifically in its handling of InRelease / Release files.

  During normal operation, debootstrap will first attempt to fetch an
  InRelease file from the repository; if this is unavailable, i.e. the
  fetch fails with 404, then debootstrap will normally fall back to
  fetching a Release file instead.  Indeed, this is the behaviour of
  debootstrap if --debian-installer is not passed on the command-line.

  debootstrap uses the wgetprogress() function to fetch these URLs.  It
  reads:

  wgetprogress () {
          [ ! "$VERBOSE" ] && QSWITCH="-q"
          local ret=0
          if [ "$USE_DEBIANINSTALLER_INTERACTION" ] && [ "$PROGRESS_NEXT" ]; then
                  wget "$@" 2>&1 >/dev/null | $PKGDETAILS "WGET%" $PROGRESS_NOW $PROGRESS_NEXT $PROGRESS_END >&3
                  ret=$?
          else
                  wget $QSWITCH "$@"
                  ret=$?
          fi
          return $ret
  }

  When the --debian-installer command-line flag is set, the first path
  of the if branch will be taken - running the output of wget through a
  pipe, so that the $PKGDETAILS command can be used to parse progress
  information provided by wget and report it to FH 3 in a format usable
  by debian-installer.

  However, if the URL passed to wget returns 404, then while the wget command
  will fail, the $PKGDETAILS command, and thus the pipeline as a whole, does not, and the wgetprocess () function thus erroneously returns success.

  As a consequence, debootstrap does not fall back to fetching and using a
  Release file as it should in this case, and the bootstrapping attempt as a
  whole fails.

  Because this is a POSIX shell-script, I don't believe there is a
  straightforward mechanism to fetch the exit status of the wget command when it is part of a pipeline.  (See: http://cfajohnson.com/shell/cus-faq-2.html#Q11). It might be possible to enhance the code invoked by $PKGDETAILS to return a fatal exit status if it does not definitely see a successful file retrieval?

  A more direct work-around is to modify this function to
  unconditionally use the second code path that does not attempt to
  invoke wget as part of a pipeline. However, this does mean that you
  lose out on intra-file download progress reporting.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debootstrap/+bug/1599459/+subscriptions