[Bug 1210503] [NEW] "allow-stderr" tests that produce stderr output always pass

[Robie Basak]

Public bug reported:

Steps to reproduce:

1. Write a test that produces stderr output, but then returns exit status 1.
2. Run adt-run against the test.

Expected results: test failure.
Actual results: test pass.

This seems to be due to the following code ("rc" contains the exit
status of the test run).

        if stab.st_size != 0:
                l = open(se_read).readline()
                l = l.rstrip('\n \t\r')
                if 'allow-stderr' not in t.restriction_names:
                    if len(l) > 35: l = l[:35] + '...'
                    t.reportfail('status: %d, stderr: %s' % (rc, l))
                else:
                    t.report('PASS')
                t._debug(' - - - - - - - - - - stderr - - - - - - - - - -')
                debug_file(se_read)
        elif rc != 0:
                t.reportfail('non-zero exit status %d' % rc)
        else:
                t.report('PASS')

** Affects: autopkgtest (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to autopkgtest in Ubuntu.
https://bugs.launchpad.net/bugs/1210503

Title:
  "allow-stderr" tests that produce stderr output always pass

Status in “autopkgtest” package in Ubuntu:
  New

Bug description:
  Steps to reproduce:

  1. Write a test that produces stderr output, but then returns exit status 1.
  2. Run adt-run against the test.

  Expected results: test failure.
  Actual results: test pass.

  This seems to be due to the following code ("rc" contains the exit
  status of the test run).

          if stab.st_size != 0:
                  l = open(se_read).readline()
                  l = l.rstrip('\n \t\r')
                  if 'allow-stderr' not in t.restriction_names:
                      if len(l) > 35: l = l[:35] + '...'
                      t.reportfail('status: %d, stderr: %s' % (rc, l))
                  else:
                      t.report('PASS')
                  t._debug(' - - - - - - - - - - stderr - - - - - - - - - -')
                  debug_file(se_read)
          elif rc != 0:
                  t.reportfail('non-zero exit status %d' % rc)
          else:
                  t.report('PASS')

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/autopkgtest/+bug/1210503/+subscriptions