Fixing rebase rather than avoiding it

[Stephen J. Turnbull]

Óscar Fuentes writes:

 > How having lots of commits that break the build or just makes the
 > application crash is not a hurdle for automatic bisection?

It's a hurdle, about 5cm high.

 > How could I teach my script to detect such cases and know that the
 > revision that breaks the build is not, precisely, the one that
 > introduced the bug I'm looking for?

It's for Mercurial, but maybe this bisect that I recently did will
help explain.  The important point is the "if make" in test.sh, which
uses the return code from make to detect a broken build.

    # Prepare the XEmacs part of the test.
    cat >test.el <<EOT
    (let (kill-emacs-hook)         ; may not be needed
      (setup-fonts)                ; host-dependent, details on request
      (kill-emacs (when (< (frame-width) 79)) 1))
    EOT

    # Prepare the test command for hg bisect.
    cat >test.sh <<EOT
    if make; then
      src/xemacs -vanilla -l ./test.el
    else
      # The value 125 tells bisect to ignore the result
      # and ignore this revision in the future.
      exit 125
    fi
    EOT

    # I don't know if this chmod is necessary,
    # I did it before I corrected a different fatal bug in my script.
    chmod a+x test.sh

    # Just do it!
    hg bisect -g 5042
    hg bisect -b tip
    hg bisect -c ./test.sh

5042 was good, as verified by "sh ./test.sh".  tip (5081) actually
includes a couple of mq patches, which I didn't intend but clearly
didn't hurt anything.  For the sake of posterity, here's the trace of
bisect results:

    Changeset 5064:501b5e84f5a7: bad
    Changeset 5053:0e803dc6f096: skip
    Changeset 5058:eb17f0c176ac: skip
    Changeset 5052:92dc90c0bb40: skip
    Changeset 5054:24372c7e0e8f: bad
    Changeset 5047:07dcc7000bbf: skip
    Changeset 5046:d4f666cda5e6: skip
    Changeset 5048:32e1ae4c1e3a: skip
    Changeset 5045:c3cc3fa503a2: bad
    Changeset 5043:d0c14ea98592: bad

Note that "skip" means that "make" failed, "bad" means that the bug
was found.

I'm not sure why there are so many skips; possibly the problem is we
run ./config.status as part of every make.  This seems to fail if
there are changes in any configure-related files.  However, it shows
that bisect works fine in the presence of such breakage.  (In your
case, running a similarly bad set of changes would take a whole day,
which would be painful.  But it would work, eventually.)