Two stage rsync mirroring of the Ubuntu archives

[Karl Tilbury]

Hello,

We have received a number of reports from Ubuntu users about getting
“file not found” errors from apt-get when updating from archive mirrors.

This error is normally caused by mirrors running an rsync update and
updating the package index files before the corresponding packages are
available.

When apt-get executes,  fetches the new package index files, apt-get and
discovers new updates and attempts to download the updates packages.  If
the packages do not yet exist,  apt receives a 404 error message from
the mirror and is unable to download the updates.

The recommended solution to this problem is to use a two stage rsync
mirror script.  The first stage fetches only the packages, while the
second stage updates the package indexes.  Updating your archive mirror
using two stage mirroring means packages should always exist before
package index files have been updated.

Ubuntu official country mirrors must use two staging mirroring.  The
Ubuntu installer defaults to using official country mirrors such as
gb.archive.ubuntu.com on installation so it is important these users are
updating from a reliable mirror source.

This is a simple example bash script demonstrating two staging
mirroring.  Please feel free to adapt it to suits your needs.


RSYNCSOURCE=rsync://archive.ubuntu.com/ubuntu
BASEDIR=/path/to/your/mirror/

rsync --recursive --times --links --hard-links \
      --stats \
      --exclude "Packages*" --exclude "Sources*" \
      --exclude "Release*" \
      ${RSYNCSOURCE} ${BASEDIR}

rsync --recursive --times --links --hard-links \
      --stats --delete --delete-after \
      --exclude "project/trace/${HOSTNAME}" \
      ${RSYNCSOURCE} ${BASEDIR}


If you require assistance implementing two stage mirroring or have other
questions related to  Ubuntu mirrors, please join us on irc.ubuntu.org
in our channel #ubuntu-mirrors.  Alternatively you can also email
mrirors at ubuntu.com for assistance.

Regards,

--
Karl Tilbury
mirrors at ubuntu.com