timeouts between ubuntu hosts because of bogus arp

[Jason Paul Joines]

    I had a very strange problem with connections timing out, getting
"connection reset by peer" errors, as well as with SSH and HTTP
connections just being unusably slow.  The end points were my Kubuntu
12.04.2 on ThinkPad T510 client system and my Ubuntu 10.04.4 Xen virtual
server hosted by Linode with about 18 hops between them.

    The problem only happened when using the wired connection on my
laptop.  When using wifi on the client, everything was fine.  The
problem turned out to be that the 10.04 Xen system for some reason
thought the laptop was on the same VLAN, even though they are not even
on the same network, and was using ARP to try and obtain the client's
MAC address for frame addressing.  The only entries in the ARP table on
the 10.04 system were a correct one for it's gateway and an incomplete
one (naturally) for the 12.04 system.  How it functioned at all, I still
don't know.  I also don't have any idea what would cause this problem.

    I've resolved it with a workaround by creating a bogus arp entry
mapping the 12.04 system's IP address to the MAC address of 10.04
systems gateway.

    Any ideas as to what's causing this?


Jason
===========