PSA: autopkgtest environment behavior change

[Bryce Harrington]

On Fri, Feb 03, 2023 at 11:38:13AM -0800, Brian Murray wrote:
> I recently discovered that autopkgtests which are getting OOM killed
> were behaving differently. Paride and I have tracked this down to a
> change in systemd's behavior[1] which results in the testbed exiting
> abnormally. I've temporarily[2] modified[3] our autopkgtest code in
> production so that this will not be considered a "testbed" failure which
> will prevent packages with autopkgtests that are OOM killed from running
> repeatedly.
> 
> I mention it because the log files for tests in this situation have
> changed and have become less informative. Looking the log file[4] for
> r-cran-rstanarm you'll see the following:
> 
> autopkgtest [18:46:11]: test run-unit-test: -----------------------]
> autopkgtest [18:46:12]: test run-unit-test:  - - - - - - - - - - results
> - - - - - - - - - -
> run-unit-test        FAIL non-zero exit status 255
> 
> Which is less helpful than the previous "Killed signal terminated"
> message. So when looking at proposed migration please keep this in mind.

Does the backend have any visibility into detecting when OOM killer was
triggered?

Bryce

> Additionally, since we are talking about tests running out of memory I
> wanted to mention that with the qemu backend it is possible to adjust
> the amount of memory your virtual machine has using the "--ram-size"
> argument. For example if I want to see if having more memory will allow
> a test to pass I'll use the following before adding it to big_packages:
> 
> autopkgtest --apt-upgrade r-cran-rstanarm --shell-fail -- qemu
> --ram-size=8192 --cpus=4 /srv/vms/autopkgtest-lunar-amd64.img
> 
> Finally, one benefit of this is that it became somewhat easier to find a
> bunch of packages that would pass if given more memory. I've
> investigated several of them and added them to 'big_packages'. If the
> developers on +1 maintenance next week could look at the following list
> it would be helpful!
> 
> amd64-igraph
> amd64-juce
> amd64-libgd2
> amd64-libthrust
> amd64-mariadb
> amd64-mariadb-10.6
> amd64-nvidia-cuda-samples
> amd64-openjdk-8
> 
> [1]
> https://github.com/systemd/systemd-stable/commit/0db0562852aafd89c84bf12cdf3de770898048d1
> [2] I've spoken with the Foundations team and they plan on uploading a
> new systemd.
> [3]
> https://git.launchpad.net/~ubuntu-release/autopkgtest/+git/development/commit/?id=842ca0f4bd78a6b0a7c1b484f7da242581651804
> [4]
> https://autopkgtest.ubuntu.com/results/autopkgtest-lunar/lunar/amd64/r/r-cran-rstanarm/20230203_185207_c0b1c@/log.gz
> 
> Have a great weekend!
> --
> Brian Murray
> 
> -- 
> ubuntu-devel mailing list
> ubuntu-devel at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel