[Bug 1075976] Re: test-suite fails in autopkgtest environment

[James Hunt]

There are a few problems here...

1) autopkgtest is running the tests are root, which isn't permissible
for Upstart.

    adt-run claims to be able to run tests as a non-root user, but that
functionality is currently broken (see Debian bug 686292).

2) autopkgtest uses libeatmydata with LD_PRELOAD which causes some tests to fail since data isn't persisted to disk.
    I've raised an MP to add a --no-eat option for auto-package-testing but aside from unsetting LD_PRELOAD, I'm not sure how
    this option could actually be specified for the Upstart autopkg build?
    See: lp:~jamesodhunt/auto-package-testing/partial-fix-for-bug-1075976

    I've also updated xnox's branch with some tweaks:
lp:~jamesodhunt/ubuntu/raring/upstart/autopkgtest

3) Some of the Upstart tests in the Ubuntu package are racy. This has been fixed upstream and the next Ubuntu release
     will thus grab those fixes but due to (2), we're hitting these races.

4) The test environment provided by adt appears to not have a controlling terminal which causes one of the test failures
    (see bug 888910 - fixed in upstream Upstart).

I wonder if we should create a "test" ("test_init" - pun intended :-)
that runs before the others that essentially sets up the required
environment to avoid duplication:

- creates user
- modifies dbus config for Upstart

In fact, since the DEP-8 spec makes no mention of test setup/teardown or
indeed the order in which tests are run but the latter does appear to be
sequential we should atleast put recommendations in place on this topic
since it can't be that uncommon a requirement.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to upstart in Ubuntu.
https://bugs.launchpad.net/bugs/1075976

Title:
  test-suite fails in autopkgtest environment

Status in Automated Testing of Ubuntu Packages:
  New
Status in “upstart” package in Ubuntu:
  New

Bug description:
  I am trying to add autopkgtests to upstart.

  $ bzr branch lp:auto-package-testing
  $ cd auto-package-testing/bin
  $ ./prepare-testbed -r quantal -d amd64
  $ ./run-adt-test -r quantal -a amd64 -b lp:~xnox/ubuntu/raring/upstart/autopkgtest upstart

  The autopkgtest in question requires root & build, the first step (building the package) fails at make check as shown in:
  http://paste.ubuntu.com/1339937/

  Please note, I have modified upstart test-suite to try dh_auto_test as
  current user & then as 'nobody'. First pass leads to three failures,
  while the second one only one. Nonetheless it is still not enough.

  Please modify adt or upstart to succeed building in such environment
  to allow running autopkgtests.

  Regards,

  Dmitrijs.

To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/1075976/+subscriptions