Confused over CIFS

[Ted Hilts]

Preston Kutzner wrote:
>
> On Jan 13, 2009, at 10:36 PM, Ted Hilts wrote:
>
>> Preston:
>>
>> I rely heavily on SAMBA and have most of the machines in my LAN cross
>> mapped with shares to each other.  I have Linux (various flavors) and
>> Windows (various flavors). Sorry for the typo. I was not complaining
>> about HP machines but rather I was complaining about MS XP Home. The
>> machine I call "Ubuntu" is used to administrate and I use it to make
>> virtual (VNC) connectionss.  For example, I am as I write this using an
>> XP Home machine called "cic2ext" and am communicating with it with
>> "Ubuntu's" keyboard and monitor. Also, at the same time I am running 6
>> machines.  None of my machines are very fast except for "Ubuntu" and
>> "misty".  Depending upon the particuar project I need a number of
>> machines to handle the data input requirements.  For example, one
>> project is to gather and archive news reports from all over the world
>> and from a number of differenct agencies in order to establish the
>> integrity (a lot of bias gets into news reports both intentionally and
>> unintentionally). One of my machines cannot handle this task by itself
>> and the overall process requires my intervention (which is time
>> consuming).  So, I end up running several machines all doing different
>> parts of the overall news reporting task.  For this particular project I
>> have to set up each machine to run the same applications and then
>> interact manually at various stages of their progress.  I used to build
>> programs to automate this overall process but ended up constantly
>> revising the programs and spending more time building the programs than
>> collecting the news.  Every time a news agency changed their web pages I
>> had to revisit my automated programs which involved my intervention any
>> way.  So I ended up using the Firefox browser with various add-ons such
>> as Newsfox to gather RSS articles and also an add-on called scrapbook
>> which captures a tree or tabbed news article.  The machine "misty"
>> handles about 300 different news sources.  Another machines called
>> "CICERO" handles about 50 different news sources, the machine called
>> "Ubuntu" handles about 100, and so on.  Most of the machines are XP and
>> they archive onto the NTFS disk share (/media/sdg1)  located on the
>> "Ubuntu" local system.  If this reliance on XP machines was not
>> necessary then there would be no problem using ext3.  I realize that XP
>> machines can access ext3 but when I researched the idea I was not
>> comfortable because of the number of XP machines involved.  Remember,
>> "Ubuntu" is dual booted with a MS XP Home OS.  I installed "Ubuntu" as a
>> experiment and fell in love with it.    So that is the story. Right now
>> there is no problem  other than the one I reported.  I'm a little
>> nervous about rsync because the files and numberous and small -- even
>> the tree files.  I don't want rsync doing anything fancy such as one of
>> it's features where it adds to an exiting file that is changed.
>> Probably this is not going to be a problem if rsync does it's thing and
>> after that does not alter a file.  I like the idea of compression to
>> make the data transfer faster.
>
> Alright, from the above, it sounds like you have One machine running 
> Ubuntu (your "Ubuntu" machine), which is sharing its Windows 
> installation as a Samba share.  If I'm incorrect about this, please 
> correct me.  All of your Windows XP boxes backup to that machine.  You 
> mention that your "Ubuntu" box is dual-boot, but it sounds like you 
> primarily, or solely use it while booted into Ubuntu.  Do you have any 
> need to have Windows installed on it anymore?  If not, you don't 
> really have any reason to keep NTFS on that machine.  You don't have 
> to use NTFS on a linux box that is sharing via Samba, even if it's 
> sharing to a network of only Windows machines.  Samba handles all 
> that, and is able to look like a Windows machine, even if it's using a 
> native Linux FS.
>
> If you're worried about preserving Windows file permissions and acls, 
> you can compile a kernel with acl support, as well as Samba with acl 
> support that will honor Windows permissions.  This still does not 
> require using NTFS on the samba server. :)  The only reason to 
> continue using NTFS on your Ubuntu box is if you still plan on 
> accessing those drives through the Windows installation *on that 
> machine*.  Otherwise, there is no need for it, and again, it might be 
> part of, if not *the* cause of your original problem copying files.
>
>>
>>
>> What would the command line look like if I used rsync instead of what I
>> did do?  I think if I use rsync I may have to install SHH for Windows on
>> all the XP Home machines as well as rsync for Windows.  I have been
>> looking at "putty" for that purpose.  Also, I think that configuration
>> files need to be created?  If so, I would appreciate some examples.
>
> I haven't used the windows rsync solutions, so I don't know exactly 
> how they work.  But, you wouldn't necessarily need SSH.  Since it 
> sounds like you're running these machines on a private network, SSH is 
> not necessary.  Rsync uses its own protocol over tcp/ip and also uses 
> its own ports.  You'd just need to make sure that the rsync server is 
> running on your Ubuntu box and that your windows boxes just use the 
> rsync client to transfer files.
>
> Regarding your statements about rsync in the previous section, what 
> exactly are you trying to accomplish with the file transfers.  Are you 
> trying to do a full copy every time?
>
> The point of rsync is to minimize the amount of data that needs to be 
> sent during the syncing process.  The first time you run it, it will 
> fully copy all files specified.  Each subsequent time it's run it will 
> copy file "deltas" or just the information that's changed in the file 
> and will update the file on the target machine to reflect what's 
> changed since rsync was last run.  You can also tell rsync to sync 
> deletes, so as files are deleted from the source machine, those 
> deletions are reflected on the target machine.  It is the most 
> efficient way, actually, of updating a large number of files on a 
> consistent basis.
>
> Here's a decent primer on how rsync actually works: 
> http://samba.anu.edu.au/rsync/how-rsync-works.html
>
> Using rsync would save time and bandwidth during your backup sessions 
> from your client machines.
>
>>
>>
>> After this particular data transfer runs properly then I need to create
>> a universal (Windows and Linux) backup so I can restore a partition or
>> entire disk. I have been researching all this but it is very time
>> consuming and it might be easier to create a shell and build into the
>> shell each piece needed  to handle the various data transfers, and
>> various backup scenarios.  I could maybe do this one piece at a time
>> instead of the whole thing in one shot. For example, to use backuppc one
>> has to involve perl, set environmental variables and set up
>> configuration files. Then the same for SHH.  Then the same for rsync.
>> And other things.
>
> Which machine will be creating this backup?  Will the Ubuntu machine 
> be doing so?  What media will you be putting these backups on?  If 
> you're simply looking to create an exact copy of a partition, you can 
> use the 'dd' command.  It will do an exact bit-for-bit copy to either 
> another block device (hard drive), or to a file.  The file can then be 
> copied anywhere you have space for it.  You need to make sure you have 
> the same amount (or more) of space available on your target media as 
> you have data on your source partition.  If your partition is 250GB, 
> you will need at least 250GB of space available on your target media.  
> Note that 'dd' will not work to back up to tape.  Also, you can't use 
> it to directly write to CD/DVD.
>
> If you're looking to back up to tape, there are some Linux solutions 
> out there, but the good ones are usually pay-for applications.
>
>>
>>
>> Thanks, Ted
>>
>>
>> -- 
>> ubuntu-users mailing list
>> ubuntu-users at lists.ubuntu.com
>> Modify settings or unsubscribe at: 
>> https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
>
Preston

You seem to have grasped most of what I said.

Once I get a universal backup set up which includes "backuppc" working I 
will probably replace the NTFS disk which we have been talking about 
(because it will be full) with a new one formatted as ext3. Several 
months ago I was told by someone on the list that I had to install a 
Windows version of ext3 on all the XP machines -- maybe I misunderstood 
them.  You seem to be saying that all I have to do is map the XP 
machines to the Ubuntu share and SMB will look after everything else.  
They are already so mapped.  So that only leaves the issue of rsync.  I 
allready read that URL article on rsync.  I have a print out "How to 
regularly backup Windows XP to Ubuntu, using rsync".  There are a lot of 
configuration steps.  Aside from that my worry is not the transfer 
process of rsync but rather this is not so much a backup transfer of 
data but an archive which gets put into physical place just once (and 
added to) so it can be referenced and used from one single place.  Every 
machine collecting data has its own local system repository which I 
destroy after a successful transfer of the data over to the "Ubuntu" 
location.  The scrapbook application has a export function that allows 
me to move data directly from the XP Pro machine (also Linux machines 
using same application) over to the mapped Ubuntu machine. HOWEVER (I'm 
not yelling) the XP Home machine called "misty" cannot make the export 
happen because it is XP Home and not XP Pro.  This "misty" machine 
cannot see the Ubuntu shares for the common workgroup. That's what led 
to this whole discussion.  I tried to get help from the list as well as 
commercial help but no one seems to understand the irregularity that is 
occuring.  Oddly enough I do have one XP Home machine called "cic2ext" 
that does not have this irregularity (it's much older) and it can see 
and access the Ubuntu share on the NTFS disk we have been talking 
about.  The XP Home machine "misty" has all the MS updates and Service 
Packs installed but this is not the case with the "cic2ext" machine.  
That's why I do not update "cic2ext" because I think the problem is 
inherent in the MS update.  BTW the dual boot XP Home / Ubuntu machine 
came into existence because that XP Home machine showed the same 
irregularity as "misty".  So that's the reason I dont' use it and 
thought I would try Ubuntu on that machine. Also, the name of the XP 
Home machine dual booted with "Ubuntu" has the name "back-up" because it 
was supposed to be the final repository not "Ubuntu".  But "Ubuntu" won 
out as the dominant computing entity because I could not get "back-up" 
to see the other machines in the LAN.  The problem with "back-up" is the 
exact same problem as with "misty"  -- they are both XP Home based and 
inherit this irregularity as I call it.

So, all that said, there would have been no "Ubuntu" machine as I now 
know it if this irregularity was solvable by me. It was a kind of 
evolution driven by the hardship imposed by this irregularity.   That 
then is the reason why I had set up "misty" the way I did using "cp", 
and  the "smbmount" generated from "Ubuntu" on a "misty" share.  By this 
means it seemed to me that the "Ubuntu" machine was pulling (by means of 
the smbmout) the "misty" data over to the "Ubuntu" share /media/sdg1 
NTFS  hard drive.  And it appears to have worked or worked to some 
degree and is the reason why I am considering your idea of using rsync.  
But I am still worried that rsync will not treat the data transfer to 
the archive  as a one shot effort but rather as a backup and may end up  
doing things to the archive files that make them unusable??? Hope that 
clarifies things?

Thanks - Ted
Watch out for typos -- you caught the last I know of where SHH was 
supposed to be SSH.