asymetric GigE speeds between Linux and FreeBSD

security security at
Tue May 22 00:05:15 UTC 2007

I've cross posted this to the freebsd-net list. I'm new to the
(k)ubuntu lists, so if this isn't the right place to post, please let me

Summary: Using iperf to measure TCP net speed between a linux (kubuntu
edgy) and freebsd box over gigE, I see significant speed difference
on the data direction. Pushing data from the freebsd box to the linux box,
I average about 500Gb/s. Pushing data from the linux box to the freebsd
box, I see about 300Gb/s. It would seem that there is something
limiting either the transmit side of the linux box or the recv side of
the freebsd box. Any suggestions on how to narrow down who's tuning
needs more work would also be welcome. Just for grins, I tried upping
the txqueuelen on the linux box to 1500, but no help.

I'd like to get closer to the theoretical speed of this link (I'd be
happy if I could get 600-700 both ways), but the real puzzler is the
difference. The client is the linux box @ and the iperf
server is the freebsd box @ . Neither box is cpu or net
busy during testing. Both systems only have one nic. I played with
larger window sizes, but it only had a minor affect. I went with the
Intel Pro/1000's based on their good rep in both the linux and freebsd

Client connecting to, TCP port 5001
TCP window size:   128 KByte (default)
[  6] local port 34788 connected with port 5001
[  6]  0.0-10.0 sec    369 MBytes    309 Mbits/sec
[  5] local port 5001 connected with port 52963
[  5]  0.0-10.0 sec    597 MBytes    500 Mbits/sec

Switch is the Netgear GS105 (5 port, supposedly wire speed, cables are
Belkin 5e), both systems are on this switch.

FreeBSD box:
FreeBSD 6.1-RELEASE-p10
1.9Ghz Athlon / 1 gig of main mem
Abit/nforce2 MB+chipset (onboard nic disabled in bios)
Intel Pro/1000GT NIC
sysctl reports:
kern.ipc.nmbjumbo16: 0
kern.ipc.nmbjumbo9: 0
kern.ipc.nmbjumbop: 0
kern.ipc.nmbclusters: 25600
net.inet.tcp.rfc1323: 1

        inet6 fe80::20e:cff:feda:1a3c%em0 prefixlen 64 scopeid 0x1
        inet netmask 0xffffff00 broadcast
        ether 00:0e:0c:da:1a:3c
        media: Ethernet autoselect (1000baseTX <full-duplex>)
        status: active

Linux box:
Linux emperor 2.6.17-11-generic #2 SMP Tue Mar 13 23:32:38 UTC 2007 i686
Soyo Dragon+ MB/ 1.9 Ghz Athlon/ 1 gig main mem.
I run at boot on the linux box
echo "10000 131072 262144">/proc/sys/net/ipv4/tcp_rmem
echo "10000 131072 262144">/proc/sys/net/ipv4/tcp_wmem
echo "131072 262144 8192000" > /proc/sys/net/ipv4/tcp_mem

/proc/sys/net/ipv4/tcp_window_scaling is 1
/proc/sys/net/ipv4/tcp_timestamps is 1

eth1      Link encap:Ethernet  HWaddr 00:0E:0C:DA:1A:3B
          inet addr:  Bcast:  Mask:
          inet6 addr: fe80::20e:cff:feda:1a3b/64 Scope:Link
          RX packets:8139539 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5638407 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:147638720 (140.7 MiB)  TX bytes:1109677958 (1.0 GiB)
          Base address:0xd800 Memory:e2020000-e2040000

Settings for eth1:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: umbg
        Wake-on: g
        Current message level: 0x00000007 (7)
        Link detected: yes

More information about the kernel-team mailing list