[Bug 1384503] [NEW] rsync fails on large files with compression

Joe Harrington jh at alum.mit.edu
Wed Oct 22 23:49:30 UTC 2014


Public bug reported:

Copying large (>10GB) files with rsync -z (compression) leads to a long
hang and eventual error after transferring part of the file.  The error
is consistent.  The file copies at normal speed until it reaches its
maximum size (1.4 GB out of 20 GB for one, 6.9 GB out of 29 GB for
another).  Then nothing happens for a while (many minutes).  Finally,
there is an error:

[....]
jh/.VirtualBox/win7/win7.vbox
jh/.VirtualBox/win7/win7.vbox-prev
jh/.VirtualBox/win7/win7.vdi
rsync: [sender] write error: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at io.c(837) [sender=3.1.0]

In this case, 6.9 GB of 29 GB transferred.  Without -z, it works.

See the following upstream report, with a comment at the end from the
rsync maintainer:

https://bugzilla.samba.org/show_bug.cgi?id=10372

According to this report, version 3.1.0 (included in 14.04) uses a
different compression package from prior versions.  Prior versions did
not have this problem for me using the same command on the same systems.
Both hosts ran Ubuntu 11.10 at the time, and all run 14.04 now, in each
case with all updates applied, Intel hardware.  Network connection
between them is gigabit ethernet through one switch.  A shell ssh
between them in a terminal works and stays up during the failure, so it
is not a network issue.  There are no relevant entries in syslog on
either machine.  There is sufficient capacity on the receiving disk.
All filesystems are ext4.

rsync command:

/usr/bin/rsync -aHSxvz --delete --stats --exclude=lost+found
--exclude=.gvfs --exclude=/nonlaptop /home/
backup.host.edu:/bu/host/home/

(yes, I changed the machine names)

Current release (both hosts):

Description:	Ubuntu 14.04.1 LTS
Release:	14.04

Current package (both hosts):

rsync:
  Installed: 3.1.0-2ubuntu0.1
  Candidate: 3.1.0-2ubuntu0.1
  Version table:
 *** 3.1.0-2ubuntu0.1 0
        500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
        100 /var/lib/dpkg/status
     3.1.0-2 0
        500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

Thanks,

--jh--

** Affects: rsync (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to rsync in Ubuntu.
https://bugs.launchpad.net/bugs/1384503

Title:
  rsync fails on large files with compression

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1384503/+subscriptions



More information about the Ubuntu-server-bugs mailing list