Ubuntu's performance : how to speed up ?

Fri Feb 18 12:56:09 UTC 2005

I have the following rsync-script.

The really cool thing about it is that it keeps more than one version
of what you backup.

So, on the backup server you get
/path/to/backup/2005-02-12
/path/to/backup/2005-02-14
/path/to/backup/2005-02-16

For example. The other cool thing is that all files that are the same
between two different backups are hardlinked to the same file. So,
even if you have 1000 MB in each backup, the only overhead you have is
the actuall changes between them.

Here is the script

#!/bin/bash

# run with -n for dryrun

backdir=/home/bagfors
remotebackdir=/home/bagfors/backup
remotehost=backup

date=$(date --iso-8601)
last=$(ssh $remotehost ls -tr $remotebackdir| grep 200 | tail -1)
if [ "$last" != "" ]; then
	link="--link-dest=$remotebackdir/$last"
fi

rsync -avb $1 -W -e ssh \
	$link \
	--exclude /.gnupg/ \
	--exclude /.ssh/ \
	--exclude cache/ \
	--exclude Cache/ \
	--exclude Media/ \
	--exclude old/ \
	$backdir $remotehost:$remotebackdir/$date

Rsync rocks!

Regards,
Erik

On Fri, 18 Feb 2005 23:09:09 +1100, Russell Cook
<ruscook_oz at yahoo.com.au> wrote:
> Hi guys,
> For those newbies, like me thinking of rsync. Here's what I've setup and
> have running at the moment. Note. I'm using the rsync daemon mode and
> haven't got this working across samba shares yet.
> 
> Each file or script below is bounded by **** filename **** so it's
> reasonably easy to piece together what I've done.
> 
> Russ's rsync setup.
> 
> The script I run from my home dir (note: remember to make executable)
> **** sync.sh ******
> #!/bin/sh
> # change -Ca to - Cav to add verbosity to output.
> 
> if [ "$HOSTNAME" != iq.sydwest ] ; then
> echo Syncing $HOSTNAME to IQ
> sudo rsync -Caz  --exclude-from /home/ruscook/sync.exclude --include "*"
> /home/ iq::home
> # echo Syncing IQ to $HOSTNAME
> # sudo rsync -Caz  --exclude-from /home/ruscook/sync.exclude --include
> "*" iq::home/ /home
> fi
> 
> if [ "$HOSTNAME" != proto ] ; then
> echo Syncing $HOSTNAME to Proto
> sudo rsync -Caz  --exclude-from /home/ruscook/sync.exclude --include "*"
> /home/ proto::home
> # echo Syncing Proto to $HOSTNAME
> # sudo rsync -Caz  --exclude-from /home/ruscook/sync.exclude --include
> "*" proto::home/ /home
> fi
> 
> if [ "$HOSTNAME" != dalek ] ; then
> echo Syncing $HOSTNAME to dalek
> sudo rsync -Caz --exclude-from /home/ruscook/sync.exclude --include "*"
> /home/ dalek::home
> # echo Syncing dalek to $HOSTNAME
> # sudo rsync -Caz  --exclude-from /home/ruscook/sync.exclude --include
> "*" dalek::home/ /home
> fi
> 
> if [ "$HOSTNAME" = dalek ] ; then
> echo Syncing linux-dist from dalek to proto
> sudo rsync -Ca /home/linux-dist/  proto::linux-dist
> echo Syncing vm from dalek to proto
> sudo rsync -Ca /home/vm/  proto::vm
> fi
> 
> if [ "$HOSTNAME" = proto ] ; then
> echo Syncing linux-dist from proto to dalek
> sudo rsync -Ca /home/linux-dist/  dalek::linux-dist
> echo Syncing vm from proto to dalek
> sudo rsync -Ca /home/vm/  dalek::vm
> fi
> **** sync.sh ******
> 
> sync.exclude is the stuff I leave out of home dir copying. Mainly for
> fear of screwing up too many settings between machines - I'm still
> learning on some of this stuff. I do force, thunderbird and mozilla to
> sync though, so my message base, bookmarks etc are duplicated across
> machines.
> As you can see from the sync.sh file above this sync.exclude file also
> resides in my home dir.
> **** sync.exclude ****
> lost+found/
> linux-dist/
> .Trash/
> .Trash-root/
> Documents and Settings/
> vm/
> tmp/
> Desktop/
> .kde/
> Cache/
> + .thunderbird/
> + .mozilla-thunderbird/
> + .mozilla/
> .*
> .config/
> .gnome/
> .gconf/
> .gconfd/
> **** sync.exclude ****
> 
> Next this rsyndc.conf file goes in your /etc directory
> Modify to suit the directory groups/parts of the tree you want to backup.
> Note I've setup rsync to allow my private class A (10.1.1.x) network to
> connect. This simplifies security but can be a risk if you're not on a
> private IP range and behind a NAT/Firewall.
> Also, you'll need to change the hosts allow to suit your IP range. The
> syntax I've used is only one way to use this option. man will give
> further details.
> **** rsyncd.conf *****
> hosts allow = 10.1.1.0/24
> use chroot = no
> max connections = 6
> log file = /var/log/rsyncd.log
> gid = nogroup
> uid = root
> 
> [home]
>    path = /home
>    read only = false
>    comment= home dir sync
> 
> [linux-dist]
>    path = /home/linux-dist
>    read only = false
>    comment = linux dists
> 
> [vm]
>    path = /home/vm
>    read only = false
>    comment = vm files
> 
> [opt]
>    path = /opt
>    read only = false
>    comment = sync opt paths
> **** rsyncd.conf *****
> 
> The /etc/defaults directory has an rsync script (put their by the system
> when rsync is installed). This needs to have the RSYNC_ENABLE line
> changed from the default of false to true.
> **** rsync ****
> # defaults file for rsync daemon mode
> 
> # start rsync in daemon mode from init.d script?
> #  only allowed values are "true", "false", and "inetd"
> #  Use "inetd" if you want to start the rsyncd from inetd,
> #  all this does is prevent the init.d script from printing a message
> #  about not starting rsyncd (you still need to modify inetd's config
> yourself).
> RSYNC_ENABLE=true
> 
> # which file should be used as the configuration file for rsync.
> # This file is used instead of the default /etc/rsyncd.conf
> # RSYNC_CONFIG_FILE=
> 
> # what extra options to give rsync --daemon?
> #  that excludes the --daemon; that's always done in the init.d script
> #  Possibilities are:
> #   --address=123.45.67.89        (bind to a specific IP address)
> #   --port=8730                (bind to specified port; default 873)
> RSYNC_OPTS=''
> 
> # Don't forget to create an appropriate config file,
> # else the daemon will not start.
> **** rsync ****
> 
> Below is an excerpt from the final file, the rsync script file in
> /etc/init.d this file also needs the RSYNC_ENABLE=false changed to
> RSYNC_ENABLE=true
> **** rsync **** EXTRACT ONLY NOT WHOLE FILE
> #! /bin/sh
> set -e
> 
> # /etc/init.d/rsync: start and stop the rsync daemon
> 
> DAEMON=/usr/bin/rsync
> RSYNC_ENABLE=true
> 
> **** rsync **** EXTRACT ONLY NOT WHOLE FILE
> 
> Finally, to start rsync open a terminal and type "sudo rsync start"
> 
> Kind Regards Russ
> www.windsorcycles.com.au
> (gedit/ubuntu/gFTP)
> 
> 
> --
> ubuntu-users mailing list
> ubuntu-users at lists.ubuntu.com
> http://lists.ubuntu.com/mailman/listinfo/ubuntu-users
> 
> 
> 
>