Filesystem duplication

[Bart Silverstrim]

Okay, here's a question for people.  It's kind of long, but please bear 
with me as I could use an idea of what direction to look in for this.

We're on a budget, of course, so that kind of limits solutions available.

I'm looking for a way to duplicate servers storage using Linux.  I'd 
like to have a way to have a file server that, when a system write 
occurs, would be written to 2 different machines.  If one dies, I can 
switch over to the other one (or use heartbeat and high availability to 
have it automatically switch).

Ideally I'd be able to have one server in one location and another 
server in another building.

I know that for some configurations of VMWare, there is a feature touted 
where you can set up (if I understand this correctly) 2 servers to guard 
against hardware failure and if one machine dies, the other one will 
pick up the first one's virtual machine load and kick over, but this 
still relies on a third computer to act as a NAS.  What happens if the 
NAS needs updating or has a failure?  Wouldn't it all die?

So I'd like to get a configuration built that would run a virtual server 
in one building and a storage system to go with it, and another server 
and storage server in another building, but that would still rely on 
data getting synced between the two storage servers.  If I don't have 
auto-migration, I could at least hope to have some form of the virtual 
drives duplicated and can manually start them if need be.

Is anything like that available?

Or am I simply going to have to start looking at only having redundancy 
on the execution server, and the NAS hardware can only be backed up in 
some other fashion?

How do other admins deal with this situation?

Are there any filesystems that are tested and reliable that work under 
Ubuntu for writing data between two machines to act as a "virtual 
server," or better yet a howto resource for HA that would allow such a 
configuration that also includes what's necessary for the cloning of the 
MAC address, the IP switchover, etc...?

And could such a configuration be done using 2 or 3 Linux systems, or 
would it also rely on dedicated special hardware/switch configurations?

Thank you,
-Bart