Help! Instable network on 6.06 based SuperMicro server with e1000 Intel driver

[Nick Webb]

Floris Vlasveld wrote:
> On 24 feb 2008, at 21:49, Aart Koelewijn wrote:
> 
>> On Sun, 24 Feb 2008 21:22:23 +0100, Floris Vlasveld wrote:
>>
>>> On 24 feb 2008, at 16:52, Nick Webb wrote:
>>>
>>>> Floris Vlasveld wrote:
>>>>> And, the output of "ethtool eth0" and eth1:
>>>>>
>>>>> Settings for eth0:
>>>>> Supported ports: [ TP ]
>>>>> Supported link modes:   10baseT/Half 10baseT/Full
>>>>>                       100baseT/Half 100baseT/Full
>>>>>                       1000baseT/Full
>>>>> Supports auto-negotiation: Yes
>>>>> Advertised link modes:  10baseT/Half 10baseT/Full
>>>>>                       100baseT/Half 100baseT/Full
>>>>>                       1000baseT/Full
>>>>> Advertised auto-negotiation: Yes
>>>>> Speed: 100Mb/s
>>>>> Duplex: Full
>>>>> Port: Twisted Pair
>>>>> PHYAD: 0
>>>>> Transceiver: internal
>>>>> Auto-negotiation: on
>>>>> Supports Wake-on: umbg
>>>>> Wake-on: g
>>>>> Current message level: 0x00000007 (7) Link detected: yes
>>>>>
>>>>> Settings for eth1:
>>>>> Supported ports: [ TP ]
>>>>> Supported link modes:   10baseT/Half 10baseT/Full
>>>>>                       100baseT/Half 100baseT/Full
>>>>>                       1000baseT/Full
>>>>> Supports auto-negotiation: Yes
>>>>> Advertised link modes:  10baseT/Half 10baseT/Full
>>>>>                       100baseT/Half 100baseT/Full
>>>>>                       1000baseT/Full
>>>>> Advertised auto-negotiation: Yes
>>>>> Speed: 100Mb/s
>>>>> Duplex: Full
>>>>> Port: Twisted Pair
>>>>> PHYAD: 0
>>>>> Transceiver: internal
>>>>> Auto-negotiation: on
>>>>> Supports Wake-on: umbg
>>>>> Wake-on: g
>>>>> Current message level: 0x00000007 (7) Link detected: yes
>>>> I have similar SuperMicro servers with no issues.  I'd look first at
>>>> the
>>>> switch this server is connect to and make sure the ports are set to
>>>> auto
>>>> negotiate just like your interfaces are.  If they are set to auto
>>>> negotiation, then I'd force them to 100/full and do the same to your
>>>> server's network cards.  I've seen speed/duplex mismatches cause  
>>>> lots
>>>> of
>>>> odd problems before.
>>>>
>>>> Aside from that, I'd be looking for hardware issues.  See anything  
>>>> odd
>>>> in the logs?
>>>>
>>>>
>>>> Nick
>>>
>>> Thanks for the tips Nick. I've first of all forced eth0 (the  
>>> 'external'
>>> interface) to operate at 100/Full. However, that didn't stop the
>>> problem. I then changed the settings so that eth0 was allowed to
>>> negotiate, but only up to 10/Full. Unfortunately, this didn't solve  
>>> the
>>> problem as well. At this time, I'm not able to force the 3com  
>>> switch to
>>> operate at 100/full only. However, it should be able to handle both
>>> 100/full and 10/full perfectly.
>>>
>>> Output of ethtool eth0 at this time:
>>>
>>> Settings for eth0:
>>> 	Supported ports: [ TP ]
>>> 	Supported link modes:   10baseT/Half 10baseT/Full
>>> 	                        100baseT/Half 100baseT/Full
>>> 	                        1000baseT/Full
>>> 	Supports auto-negotiation: Yes
>>> 	Advertised link modes:  10baseT/Full
>>> 	Advertised auto-negotiation: Yes
>>> 	Speed: 10Mb/s
>>> 	Duplex: Full
>>> 	Port: Twisted Pair
>>> 	PHYAD: 1
>>> 	Transceiver: internal
>>> 	Auto-negotiation: on
>>> 	Supports Wake-on: umbg
>>> 	Wake-on: g
>>> 	Current message level: 0x00000007 (7) Link detected: yes
>>>
>>> Hardware issue is of course always possible. However, since both eth0
>>> and eth1 display this problem, that seems quite unlikely.
>>>
>>> Do you or anybody else have any other suggestion?
>>>
>>> Floris.
>> With such a problem I would always check route. It has happened to me
>> that there are 2 default routes or none at all. In both cases you will
>> have problems with your network.
>>
>> Aart
> 
> Thanks for your reply Aart. How exactly would I check that? What do I  
> look for?
> 
> Output of "route" is as follows:
> 
> Kernel IP routing table
> Destination     Gateway         Genmask         Flags Metric Ref     
> Use Iface
> 80.69.93.0      *               255.255.255.128 U     0      0         
> 0 eth0
> 10.0.0.0        *               255.255.255.0   U     0      0         
> 0 eth1
> default         10.0.0.1        0.0.0.0         UG    0      0         
> 0 eth1
> default         80.69.93.1      0.0.0.0         UG    0      0         
> 0 eth0
> 
> 

As Aart said you have two default routes setup, that won't work.
Basically if there is no route matching the host you want to connect to,
it will use the default.  In the output here you have both 10.0.0.1 and
80.69.93.1 as default routes.  I think you only want the later as
default, as it can route you to the internet.  Then the 10.0.0.1 gateway
is only used for 10.0.0.x address, which is what you want.

This command should fix this:

sudo route del default gw 10.0.0.1

Then you'll need to update the appropriate /etc files to make it stick. 
  I don't have this kind of a setup on ubuntu, so I'm not sure how to do 
this.

Nick