[Bug 2039441] [NEW] "System is booting up. Unprivileged users are not permitted to log in yet." causes wait subcommand to fail

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Ubuntu Foundations Team Bug Bot (crichton):

[ Impact ]

The bug breaks `uvt-kvm wait` promise that the virtualized system is up and running. `uvt-kvm` naively waited for the ssh server to be up, but if the
ssh server is up before authentication (pam notably) is ready, the system is still not ready to use.
For that reason, all automation relying on it breaks and report failures blaming the underlying image instead of waiting for it to be really ready.
This fix will make `uvt-kvm wait` retry in case the login fails.

[ Test Plan ]

Use the patched uvt-kvm to start then wait for this image to come up:
http://cloud-images.ubuntu.com/releases/focal/release-20231011/ubuntu-20.04-server-cloudimg-amd64.img
This image was know to trigger this error, so verify ten times we can: start it, wait then ssh without any failure.
Verify also this with a previously working image such as http://cloud-images.ubuntu.com/daily/server/focal/20231003/focal-server-cloudimg-amd64.img

[ Where problems could occur ]

The retry mechanism could be triggered for other reasons than login
error and slow down the whole process.

[ Original Bug Report ]
After doing `uvt-kvm wait`, we can expect to be able to ssh into the VMs.
That's not always the case as the ssh port can be up before PAM is setup:
`System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).`
This means that subsequent programs can't rely on `uvt-kvm wait` to know if the system is up, which defeats the purpose of this function and drives the complexity up in highly automated environment.

Personally, I see two ways to fix the wait to handle this case:
- Change the behavior of the created VM to avoid this edge case.
- Makes `uvt-kvm wait` smarter by actually establishing a communication to check if we really can login.

The last option seems less intrusive but will make the library more complex.
I'm not convinced that would be a reasonable default or would be better as an option to `uvt-kvm wait`.

** Affects: uvtool
     Importance: Undecided
         Status: Fix Committed

** Affects: cloud-init (Ubuntu)
     Importance: Undecided
         Status: Fix Committed

** Affects: uvtool (Ubuntu)
     Importance: Undecided
         Status: Fix Released

** Affects: cloud-init (Ubuntu Focal)
     Importance: Undecided
         Status: Fix Committed

** Affects: uvtool (Ubuntu Focal)
     Importance: Undecided
     Assignee: Agathe Porte (gagath)
         Status: Triaged

** Affects: cloud-init (Ubuntu Jammy)
     Importance: Undecided
         Status: Fix Committed

** Affects: uvtool (Ubuntu Jammy)
     Importance: Undecided
     Assignee: Agathe Porte (gagath)
         Status: Triaged

** Affects: cloud-init (Ubuntu Lunar)
     Importance: Undecided
         Status: Fix Committed

** Affects: uvtool (Ubuntu Lunar)
     Importance: Undecided
         Status: Won't Fix

** Affects: cloud-init (Ubuntu Mantic)
     Importance: Undecided
         Status: Fix Committed

** Affects: uvtool (Ubuntu Mantic)
     Importance: Undecided
     Assignee: Agathe Porte (gagath)
         Status: Triaged


** Tags: patch regression-update
-- 
"System is booting up. Unprivileged users are not permitted to log in yet." causes wait subcommand to fail
https://bugs.launchpad.net/bugs/2039441
You received this bug notification because you are a member of Ubuntu Sponsors, which is subscribed to the bug report.