Download Speed with Kubuntu Karmic 64bit

[Reinhold Rumberger]

On Saturday 30 January 2010, Steven Vollom wrote:
> Trim again
> 
> > (I haven't read the whole thread and am hoping that this isn't
> > too redundant.)
> 
> So far, it seems the most informative.

:-)
I'd like to point out that most of my remarks were meant to be pretty 
generic, as you had stated that you would like to learn about that 
kind of stuff. I wasn't directly replying to anything you wrote with 
that first paragraph.
I'm also far from an expert on hardware - all I know I got from two 
lectures at university. I'm a software guy. Luckily, a lot of the 
copying has to do with software, so I know a fair amount about these 
things...

One more thing about ext4: they integrated some ideas into the design 
meant to speed up operations in general. One of them is that ext4 
mostly operates in RAM while it can - the actual writeback happens 
when there isn't too much going on. I'm not sure whether Ubuntu 
preserved this behaviour, as it causes a lot of problems on unstable 
systems and used to cause a fair amount of data loss when ext4 was 
still young. Because of this, the developers added a switch to make 
ext4 emulate old ext2/3 behaviour in this regard.
If this feature is active on your system (I forget how to check for 
it; I believe it was called "delayed write" or something similar), 
you may experience random spikes and slowdowns in your transfers...

<snip>

> From previous computers, the amount of ram at 4gb is so great that
> I hadn't considered a shortage or cache shortage.  I have an AMD
> quad 9600 with, I believe 512mb of cashe for each of four
> processors and a 20gb swap file for my own reasons.  So, the
> thought of running out of cache or ram hadn't hit me yet.

If you're running firefox, that is a memory hog if there ever was 
one. plasma-desktop also eats a lot of RAM. I have 3gb RAM and find 
that at least half of that is in use most of the time. But then I 
also use a lot of widgets.
Most of the time, my free RAM is enough to make file transfers 
*really* fast. When the files exceed 2gb, however, the transfer will 
slow to a crawl at some time.

> Both my HDD's are SATA with 600gb vacant on one and 400gb unused
> on the other, so I don't think there is any shortage of space,
> and perhaps the reason I get such wonderful speeds most of the
> time.  Each has cache, but I can not remember the amount right
> now.

Since the disk cache is relatively small, it won't have that great an 
impact on large transfers...

<snip>

> > Since KDE starts a new copy
> > operation for every file and does a lot of "useless" work in
> > between operations (for progress information), copying a large
> > amount of small files will take much longer using the KDE copy
> > than using the cp command.
> 
> I am just learning to use the konsole more.  I will use the cp
> command from now on, if it is faster and/or better.

I just recently had to copy a directory with a huge amount of small 
files. I first made the mistake of trying to use konqueror to do the 
transfer. After it hadn't even passed the 50% mark after five 
minutes, I cancelled and used cp. That finished within two minutes...

> > This is partially due to the buffering mentioned above. Also,
> > obviously, the target drive's write speed is a limiting factor.
> 
> Both drives are Maxtor SATA's a 1.5tb and a 500gb.  The 500gb is
> the backup drive and most large file transfers take place from
> the larger to smaller drive.  Both have the same specifications,
> as i recall.  I am going to check that.

Well, the reading speed is still going to be greater than the writing 
speed. ;-)
You can use hdparm to do some benchmarking and tweaking.

> > > I have read that ext4 is not stable;
> > 
> > That is incorrect. It isn't tested as thoroughly as its
> > predecessors, but it most certainly is stable (there was some
> > patch in the Ubuntu 9.04 kernel which rendered deleting
> > unstable, but that was an exception).
> 
> I have been successfully using ext4 for over 6 months now with no
> negative impact that I am aware of.  Perhaps my experience is
> useful for the developers.  I transfer lots of large files.

It was a pretty rare bug for most of us. It seemed to be some race 
condition as I recall. It only hit me once or twice, but it cost me a 
few gb of data... :-/

> > There is a lot of theory behind the inner workings of a
> > computer, and you don't always need to use something to be able
> > to apprehend problems.
> > Also, not using something because it's not well-tested is a
> > *very* valid reason and the one I've come across most often.
> 
> I am retired.  Most of my joy is derived from learning to use and
> using a computer.  I don't even mind crashing and re-installing,
> as long as I am not unable to use my computer for too long a
> period of time.

Then those reasons will likely not apply to you - they're mostly 
meant for newbies who would be raising a tantrum if anything went 
wrong, anyway.

I, for my part, would be pretty annoyed if a day's work vanished 
every now and then. This kind of makes you a little more careful when 
choosing the FS for your production system... :-)

  --Reinhold