rsync progress

[Johnny Rosenberg]

2013/9/29 Tommy Trussell <tommy.trussell at gmail.com>

> On Sat, Sep 28, 2013 at 1:47 PM, Johnny Rosenberg <gurus.knugum at gmail.com>wrote:
>
>> 2013/9/28 Johnny Rosenberg <gurus.knugum at gmail.com>
>>
>>>
>>> So far I have almost never got any answers to anything I asked here, so
>>> here is the next exciting question:
>>>
>>> I have a script that runs rsync for backup from my internal drive to an
>>> external one. It works great, but it would be nice to see the progress. It
>>> doesn't necessarily need to be based on file size, I just want to get some
>>> kind of hint what's going on.
>>>
>>> So I searched around and ended up with something like this, just for
>>> testing if it works:
>>>
>>> LineCount=$(find /Some/Folder | wc -l)
>>> rsync -anuv /Some/Folder /media/Backup/AnotherFolder/ | pv -l -e -p -s
>>> ${LineCount}
>>>
>>> (pv – Pipe Viewer – needs to be installed first: apt-get install pv)
>>>
>>> What happens in my case is that all those lines from rsync are scrolling
>>> and the last line is:
>>> [===================================================================================================================>]
>>> 100%
>>>
>>> Not very useful information if you ask me.
>>> I expected no lines scrolling from rsync, but only a ”progress bar” and
>>> the progress in % displayed. What am I missing?
>>>
>>
>> I think I found the answer myself in a video tutorial on YouTube…
>> I can just throw the output away:
>> LineCount=$(find /Some/Folder | wc -l)
>> rsync -anuv /Some/Folder /media/Backup/AnotherFolder/ | pv -l -e -p -s
>> ${LineCount} > /dev/null
>>
>> Then the 100% thing is all I see. It seems like it doesn't update very
>> often, that's why I only see 100%. I tried rsync with a much bigger folder
>> and now I got different numbers, and it seems like they are displayed every
>> second or so.
>> Then I found pv's -i option:
>>
>> LineCount=$(find /Some/Folder | wc -l)
>> rsync -anuv /Some/Folder /media/Backup/AnotherFolder/ | pv -l -e -p -s
>> ${LineCount} -i 0.1 > /dev/null
>>
>> Now the numbers appear ten times per second. Nice!
>>
>>
> You said you were using scripts, but if you decide you want to try your
> hand at a graphical version, you might tinker with Zenity. Here's a place I
> found (with a search) where someone was describing how to use Zenity's
> progress bar, and it includes an example showing it with rsync.
>
> http://www.davidverhasselt.com/2008/12/24/zenity-rsync-and-awk/
>
> As you can see, it uses the --progress option in rsync and an awk script
> to convert it to something Zenity could use. A person in the comments also
> shared his version using perl instead of awk.
>

That's odd, since --progress did nothing in my case, while other's stated
that --progress only displays progress for each file, which is not what I
want. Some people (on different forums that I found by searching) said that
a new version, not yet stable, has a feature for displaying progress of the
whole job, and maybe I will use that in the future.

Even if I didn't mention it, I display the progress in a graphical window,
however not using Zenity but rather Yad, which is a fork of Zenity. Why?
Because Zenity has a nasty bug that is probably fixed, but not in the
version in the software centre. The bug doesn't affect the progress bar,
but I prefer using the same application for the same kind of task, so I use
Yad for everything graphical in my Bash scripts. But that doesn't matter
much, they do the same thing, but the syntax is somewhat different in some
cases. And Yad can do more stuff, like having several different kinds of
elements in the same window. Not as advanced as GTK-dialog, but more
advanced than Zenity.

I still don't have the final solution to my problem, but currently I do
something like this:
rsync -auv --exclude-from "${ExcludeList}" --delete-during
--delete-excluded --log-file "${LogFile}" "${SorceFolder}"
"${DestinationFolder}" | pv -len --size "${CountLines}" --interval 0.1 2>&1
>/dev/null | yad --progress --title="${Title}" --auto-close

I calculate CountLines by running the same rsync command with -n (--dry-run
– same thing).
I have some new ideas though, that I want to test before I tell anyone… :P


Johnny Rosenberg



>
> You might find some awk or perl piped transformations useful regardless.
>
> --
> ubuntu-users mailing list
> ubuntu-users at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-users/attachments/20130929/6687abe7/attachment.html>