[Bug 2000874] Re: Ubuntu iso defeats the purpose of live media

Tue Jan 3 11:16:23 UTC 2023

Okay, I have now had a chance to download and look at the source code
for casper and klibc-utils (the current versions in the 22.10 repos),
and I think I have a good understanding of how this all went wrong,
which I will explain in the next few paragraphs.

Yes, I didn't mention a particular ISO because these problems affect
casper in general, not one particular ISO.  The ISO I happened to be
using was the Kubuntu 22.10 installation DVD.

Before looking at the code and finding out what happened, I was under
the impression that casper just didn't give a darn what device it put
the "writable" partition on.  Correct me if I'm wrong, but I now gather
that the INTENT is for casper to put the "writable" partition only on
the device from which it has booted.  Is that right?  It still
aggravates me that it is done without any user interaction/permission,
but if the INTENT was to only affect the device on which is the ISO
being booted from, that is better than simply willy nilly picking any
place.  Unfortunately, through stupid assumptions in the casper code, it
CAN pick a device other than the current boot device.

The reason for this (and again, please correct me if I'm wrong) is that
casper starts out not having a clue where it booted from, so it just
gets a list of all block devices that might could be it and starts using
heuristic tests to attempt to find one that looks right.  As soon as it
finds one that seems to quack like the duck it's looking for, it picks
that one.  But this selection is only by guessing and making
assumptions, not by actually knowing.  I see there's even a comment in
one section admitting, "This is an ugly hack situation, the block device
has an image directly on it.  It's hopefully casper, so take it and run
with it."

In making these stupid assumptions, casper also takes the shortcut of
picking the first thing it guesses to be correct.  So in the event that
there are multiple devices on the system that WOULD match casper's
tests, it won't even get around to considering any but the first, nor
wondering whether it screwed up in its assessment of which device it was
running from.  This is just asking for problems.  It's not at all an odd
situation that someone would happen to have multiple devices with casper
ISOs on them, even perfectly identical images.  Let's say I write an ISO
to two USB sticks, or to a DVD and USB stick, or to an SD card and a DVD
or whatever combination.  If casper identifies the wrong device as the
one from which it's running, it MIGHT "work" (as in boot), especially if
the images are identical, but the behavior is still WRONG and against
intent, particularly when it ends up writing persistence data onto the
wrong device.  Because then I could boot from the USB stick, thinking my
persistent data is written there, and expect to have it there if I take
my stick and boot from it on another computer, but it won't be there; it
will be on the other device that just happened to be in the system at
the same time.

And, importantly, it's not just the location of the "writable" partition
that is affected.  Casper will at the same time fail to correctly
identify the device it is supposed to be booting from.  It mounts the
wrong partition but assumes it to be right because it passes certain
tests.  This explains not only why casper made the "writable" partition
on my internal HDD (even though I'd booted from DVD), but also explains
why it failed to boot when that HDD was in the machine, because it
identified my entire HDD as a casper partition and mounted it.  My HDD
is NOT a casper partition, thus it failed to boot.

You may be wondering at this point why it thought my internal HDD was a
casper partition.  Here's the story:  When I was installing this system,
I had to juggle some things around and had first written the
installation ISO to my main HDD, booted from it, then made and booted
from another device instead so I could then install the OS on the HDD.
My HDD is correctly partitioned with a hybrid protective partition table
in the MBR and several GPT partitions.  Immediately after my first GPT
copy is a small 3-MB partition which I reserved for a BIOS bootloader,
in case I ever need to boot this on an older machine (of which I have
many).  So far that partition has not been written to, because this
system uses EFI.

So as casper is filtering through all the block devices on the system,
trying to find something it thinks is correct, it asks fstype (from
klibc-utils) to tell it what is the file system of /dev/sda.  Of course
that's a stupid question, and it gets a stupid easy answer: "iso9660"
simply because it has "CD001" at 32 KiB in from the beginning of the
drive.  Casper says, "oh good, I know what to do with that" and it
continues.  Next it mounts /dev/sda as type iso9660, and because the
first 3 MB of the original ISO is still there, it thinks everything is
peachy.  Now it's sure it's found the right device, so it unmounts it,
creates and formats the "writable" partition on it, and then remounts
/dev/sda according to its original idea that this must be the casper
device.  And of course it can't boot from there, because it was never
supposed to boot from there to begin with.

How can this mess be fixed?  Well, ideally, this should have been
engineered from the beginning to have a way to KNOW (not guess) what
device it actually booted from.  This business of it having to cycle
through every device on the system like the ugly duckling asking "Are
you my mother?" is pure madness.  If for some technical reason this is
impractical, casper needs to at least go ahead and "ask" ALL the
potential mothers whether it belongs to them and then face an
epistemological crisis when the search turns up more than one device.
It would still be a hack, but it would be a less error-prone hack.

As it is, the entire casper package is EXTREMELY error-prone, not only
because of such design failures, but because the whole thing is a morass
of poorly documented shell scripts, ready to fail or act erroneously if
any of the many utils being leveraged happens to output things in a
slightly different way from what is expected.  I guess that's the sort
of spit and bailing wire holding together much of the Linux experience,
but it's just rather disturbing when you look under the hood.

Please let me know if you have further questions.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to casper in Ubuntu.
https://bugs.launchpad.net/bugs/2000874

Title:
  Ubuntu iso defeats the purpose of live media

Status in casper package in Ubuntu:
  Incomplete

Bug description:
  The whole purpose of live media is to:
  1) Be able to boot into an operating system REGARDLESS of the state of the data on an internal hard drive.
  2) Be able to run an alternative operating system WITHOUT making any changes to the existing system.

  Casper (or whatever related components of Ubuntu iso) violates both
  these purposes.

  1) I've had the live DVD unable to boot simply because of minor
  problems on my installation on the HDD.  (Took the HDD out of the
  computer, and it booted fine -- then I could connect the HDD via a USB
  adapter to the running live system.)

  2) The live DVD seems to think it owns your system and can just start
  making partitions wherever the heck it pleases without asking.
  Namely, "writable", which it seems to use for stuff like log files
  which I don't want to keep anyway!  This is the sort of crap we Linux
  users left Microsoft years ago to avoid.  You don't own my system, you
  don't get to unilaterally decide to scoop up the remaining 2/3 of my 1
  TB HDD to use it for some damn log files.  All this happens before the
  user is given any options whatsoever, and the options that do show
  suggest you can "try" it before making any changes to your system,
  which is false.

  The log files on "writable" are not useful, and even if they were, you
  shouldn't need hundreds of GB for them!

  Although it will definitely use it ALL if given half a chance.
  Example:  I recently booted a computer with the iso written to a 64-GB
  usb stick.  Thankfully in that case it made the writable partition
  fill the remainder of that stick INSTEAD of screwing with other hard
  drives (not that that is acceptable either, just preferable).  But due
  to some kind of hardware incompatibility, the entire 64-GB stick
  became filled with repetitious error messages in just a few minutes of
  booting, after which the system became unstable or unresponsive.  The
  only way I could get the live system to "work" was by killing the
  rsyslogd process as soon as I could get to a command prompt, and then
  delete the several GB of crap that had accumulated in the minute it
  took to boot.

  Maybe you'll say that those log files are important to find out why the live media doesn't boot.  Two things:
  A) In the scenario described above (1), the live DVD didn't write any files to the 'writable' partition it made and formatted, because it never made it that far.
  B) In testing something before release (e.g. alpha testing), that might be fine to unilaterally write log files, but if it doesn't boot after release, that's not the time to be writing these log files.  Simply fail to boot, and let the user find something else that does boot instead.  Unilaterally adding partitions to users' systems (when they only wanted to try Ubuntu) is unacceptable.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/casper/+bug/2000874/+subscriptions