NFS performance (can't use "async")

[Steve Flynn]

On 19 July 2012 12:04, Wipe_Out <wipe_out at users.sourceforge.net> wrote:

> Both are exported and mounted NFSv3 and looking at the /etc/exports files of
> the two systems I couldn't see anything significantly different between
> them..

No surprise there. For reference, we export NFS mounts like so:

/some_filesystem -public,sec=sys,rw,sec=dh:krb5:krb5i:krb5p,rw

and we mount with

/some_filesystem:
        dev             = "/client_data"
        vfs             = nfs
        nodename        = one_of_our_aix_servers
        mount           = true
        options         = bg,hard,intr,sec=sys:dh:krb5:krb5i:krb5p
        account         = false


> Using the exact same hardware so the NIC's, cables and switch is the same..
> The network aspects of the kernels may be tuned differently but not sure how
> to compare these..

(I'm no expert so take this with the usual grain of salt)

I'd start with an eyeball of the output from "ifconfig -a" on both
systems. Make sure they both look sensible and that one of them isn't
configured to route all traffic through some flaky old F5 firewall in
the basement, by accident.

Providing both installations look the same, have the same MTU,
sensible error levels for Tx and Rx after a test session and so forth
it's time to start poking around in /proc to compare tuning values.
This is where I can offer little, as I have no Linux or FreeBSD
systems to hand - just AIX which isn't going to help you much.

This /might/ help: http://nfs.sourceforge.net/nfs-howto/ar01s05.html


Out of interest, what are you using to generate the load on the NFS
filesystem and what are you using to measure the number of FSYNCS? Are
you sure that the two are equivalent under both installations (e.g.
bonnie on Linux is running with a 64K chunk of data to write to the
NFS filesystem whereas whatever you're using on FreeBSD is running
with a 2K chunk)... I'm struggling to come up with a plausible reason
for such a massive difference in sync rates between two installations
on the same kit, talking to the same filesystem.

-- 
Steve

When one person suffers from a delusion it is insanity. When many
people suffer from a delusion it is called religion.