Merging two linux partitions

[Liam Proven]

On Tue, 14 May 2019 at 16:01, Ken D'Ambrosio <ken at jots.org> wrote:
>
> IDLE MUSINGS AHEAD -- NO PRACTICAL APPLICATION
>
> So, I got to thinking about how one *would* merge two partitions... and
> I don't really have a good answer.  I mean, for example, how do you deal
> with identically-named files/directories?  Which made me think the whole
> thing was simply impossible, when it occurred to me that there were
> "union" FS options which, logically, merged multiple partitions.  Just
> wondering if anyone knows how *those* dealt with identical filenames and
> any other obstacles?

PartitionMagic could do it.

If, for example, you merged drives C, D and E, you ended up with a big
drive C which contained subfolders called "\D\" with the full contents
of D: and "\E\" with the full contents of E:

It was then up to you to move stuff around to sort it.

However, the thing is this:

When you move from one drive to another drive, including separate
partitions, the OS must copy the data from source to destination, then
when it's copied, remove the original file... then repeat this for
every file. This is unavoidably slow. It applies even on the same
physical drive, if there are multiple partitions.

But if you move a file from one folder to another folder in the same
partition, on any modern filesystem, the OS can just rename the file
from

/data/my/old/file

... to...

/data/my/new/file

The actual contents of "file" don't move. So it's very, very fast.

So cleaning up the folders left by a PQMagic partition merge was quite
quick. It was the merge that took hours. It copied as much data as
would fit, shrank D: as much as possible by moving the start, enlarged
C: and then copied some more...  and repeat. This could be a *very*
lengthy process.

This kind of thing is the reason that logical volume management systems exist:

https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)

LVM is complicated and hard to understand. If the above article makes
little sense, don't blame yourself. For standalone workstations, I
recommend avoiding it.

So, there's LVM, then on top of the LVM space, you have partitions.
Those are formatted with a filesystem, such as ext4, or older
enterprise filesystems from old commercial Unixes, such as JFS (from
IBM's AIX and OS/2), or XFS (from SGI IRIX).

https://en.wikipedia.org/wiki/XFS

https://en.wikipedia.org/wiki/JFS_(file_system)

Fedora enables LVM by default which is just one reason I avoid Fedora.

Then to make matters worse, there are filesystems which support
"subvolumes" inside a partition, e.g. Btrfs.

https://en.wikipedia.org/wiki/Btrfs

Btrfs is the default FS of SUSE Linux.

Then you have subvolumes inside partitions on top of LVM volumes on
top of disks, and personally it all makes my head spin.

*Because* LVM is hard, and its functionality overlaps with
partitioning, there are projects that try to merge them.

For Linux, there was EVMS:

http://evms.sourceforge.net/

Unfortunately, it did not catch on, so we have LVM instead.

https://lwn.net/Articles/14816/

https://unix.stackexchange.com/questions/22885/is-there-a-more-modern-or-more-popular-version-of-evms2

RH does not support Btrfs. However, because it wants some of the
features of Btrfs, RH is now building its own new combined logical
volume manager / partitioner / filesystem, Stratis:

https://stratis-storage.github.io/

Stratis combines an LVM layer with the XFS filesystem.

I have heard comments that Stratis is in effect re-creating a subset
of the functionality of EVMS.

This is a very typical Linux development path.

The richest filesystem/volume manager from commercial Unix is ZFS,
from Sun (now Oracle) Solaris.

https://en.wikipedia.org/wiki/ZFS

Like JFS and XFS, ZFS is now open source. However, under a licence
that is incompatible with the Linux kernel's GPL licence.

So you _can_ compile a Linux kernel with built-in XFS, but it violates
the licence.

However, Ubuntu has found a way around this, with ZFS being a loadable
module (AIUI) that isn't part of the kernel itself.

(AIUI. IANAL. Clarification welcome.)

Ubuntu Server offers ZFS instead, in place of Btrfs in SUSE or Stratis
in Fedora (or XFS in all of them).

ZFS can replace the LVM _and_ also ext4/XFS/JFS, and therefore Stratis
too, but neither SUSE nor RH will bundle ZFS because of licence
concerns.

Apple _was_ going to bundle ZFS but it too decided the licensing was
too tricky and it has developed its own system, APFS. But then Apple
no longer is trying to compete in the server market.

https://en.wikipedia.org/wiki/Apple_File_System

Yes, it is confusing. Yes, it is a mess. Yes, there are too many standards.

https://xkcd.com/927/


-- 
Liam Proven - Profile: https://about.me/liamproven
Email: lproven at cix.co.uk - Google Mail/Hangouts/Plus: lproven at gmail.com
Twitter/Facebook/Flickr: lproven - Skype/LinkedIn: liamproven
UK: +44 7939-087884 - ČR (+ WhatsApp/Telegram/Signal): +420 702 829 053