Multiple Inheritance and super() (was Re: [Merge] lp:~vila/bzr/389648-hook-calls-base into lp:~bzr/bzr/trunk)

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Vincent Ladeuil wrote:

...

>     jam> I should comment that using "super()" is not always
>     jam> correct anymore.
> 
> 
>     jam> To start with, it only matters when you have multiple
>     jam> inheritance. But even more importantly, python 2.6
>     jam> 'broke' super(...).__init__() if there are any
>     jam> arguments.
> 
> I disagree. python-2.6 is just stricter and catch bogus calls.
> 
>     jam> Because object.__init__() no longer allows
>     jam> arguments. (So as near as I can tell, there is no way to
>     jam> safely __init__ a multiple inheritance structure that
>     jam> wants arguments passed as part of init.
> 
> Yes there is, you handle arguments where they are expected and
> you don't pass them where they are not.

So I'll expand a bit more on this later. But if you are going to "pass
them where they are expected, and not where they are not", then you
can't use super() [because what that actually represents is not static,
but based on the inheritance heirarchy]. And with python 2.6 you can't
use **kwargs to allow any arguments to __init__ and have them passed to
whoever actually cares about them.

> 
> The cases I had to fix when I addressed python-2.6 compatibility
> weren't that complex once I understood what was going on.
> 
> And as I remember the only case that seemed controversial at the
> time was indeed an abuse.

So imagine the case:

class Mixin1(object):

  def __init__(self, keyword1=None):
     self.keyword1 = keyword1

class Mixin2(object):

  def __init__(self, keyword2=None):
     self.keyword2 = keyword2


class Mixed(Mixin1, Mixin2):

  def __init__(self, keyword1=None, keyword2=None):
    ?

In python2.5 you could have written this as:

class Mixin1(object):
  def __init__(self, keyword1=None, **kwargs):
     super(Mixin1, self).__init__(**kwargs)
     self.keyword1 = keyword1

class Mixin2(object):
  def __init__(self, keyword2=None, **kwargs):
     super(Mixin2, self).__init__(**kwargs)
     self.keyword2 = keyword2

class Mixed(Mixin1, Mixin2):
  def __init__(self, keyword1=None, keyword2=None, **kwargs):
    super(Mixed, self).__init__(keyword1=keyword1, keyword2=keyword2,
                                **kwargs)

And it would have worked, and the parameters would have been passed to
the appropriate base class.

In Python2.6 this fails because object.__init__() doesn't take any
parameters. So instead Mixed has to be:

class Mixin1(object):
  def __init__(self, keyword1=None):
     self.keyword1 = keyword1

class Mixin2(object):
  def __init__(self, keyword2=None):
     self.keyword2 = keyword2

class Mixed(Mixin1, Mixin2):
  def __init__(self, keyword1=None, keyword2=None):
    Mixin1.__init__(self, keyword1=keyword1)
    Mixin2.__init__(self, keyword2=keyword2)

But note that this is explicitly *not* done using super().

Now *if* you didn't plan ahead for multi-inheritance by allowing
**kwargs, then things break. And you can work around this by doing:

class KwargsInit(object):
  def __init__(self, *args, **kwargs):
     # Don't call super() here, it breaks things in python2.6
     pass

class Mixin1(KwargsInit):
  def __init__(self, keyword1=None, **kwargs):
     super(Mixin1, self).__init__(**kwargs)
     self.keyword1 = keyword1

class Mixin2(KwargsInit):
  def __init__(self, keyword2=None, **kwargs):
     super(Mixin2, self).__init__(**kwargs)
     self.keyword2 = keyword2

class Mixed(Mixin1, Mixin2):
  def __init__(self, keyword1=None, keyword2=None, **kwargs):
    super(Mixed, self).__init__(keyword1=keyword1, keyword2=keyword2,
                                **kwargs)

That works because the diamond is rooted at KwargsInit, and not
'object'. However, all of your classes in your multi-inheritance
hierarchy that want to use **kwargs style initialization have to inherit
from KwargsInit and *not* object.

Anyway, this is a separate discussion, so I moved it to the regular list.

> If you can provide examples in the actual code base where calling
> super().__init__ will be bogus, I'll be happy to revise my
> opinion.
> 
>         Vincent

Only that in the actual code base we've side-stepped the issue by not
really using multiple inheritance, and not passing arguments via
__init__. (We *do* pass arguments via __init__ but only in cases where
we aren't using multiple inheritance, which are cases where super()
provides *no* benefit over just calling BaseClass.__init__(self, ...))

I agree it would be good to come up with a reasonable standard across
the code base. I think that

a) super() is meant to help in the multi-inheritance case, otherwise it
is just the same as directly calling BaseClass.__init__().

b) You can't pass arguments to super().__init__ if there is a chance to
actually get multiple inheritance. In the case above, if Mixin1 didn't
call super() then Mixin2.__init__ would not get called.


How about a different structure.

class Mixin1(object):
  def __init__(self, verbose):
     self.verbose = True
     if verbose:
       print "I'm Mixin1 in verbose mode"

class Mixin2(object):
  def __init__(self, verbose):
     self.verbose = True
     if verbose:
       print "I'm Mixin2 in verbose mode"

class Mixed(Mixin1, Mixin2):
  def __init__(self, verbose):
   ???

If you just use super as:
class Mixed(Mixin1, Mixin2):
  def __init__(self, verbose):
   super(Mixed, self).__init__(verbose)

Then Mixin2.__init__ never gets called. You need to change Mixin1 to:
class Mixin1(object):
  def __init__(self, verbose):
     super(Mixin1, self).__init__(verbose)
     self.verbose = True
     if verbose:
       print "I'm Mixin1 in verbose mode"

however, if you *do* that, it breaks in python2.6 because
object.__init__ doesn't allow arguments to be passed.

You basically have to create a new AbstractBaseClass for
multiple-inheritance which replaces 'object' as the root of your
diamond, and *doesn't* call super().__init__() itself.

I suppose this is consistent with the behavior for non-__init__ functions:

class Mixin1(object):
  def do_foo(self, arg):
    super(Mixin1, self).do_foo(arg)
    print "I'm Mixin1: ", arg

class Mixin2(object):
  def do_foo(self, arg):
    super(Mixin2, self).do_foo(arg)
    print "I'm Mixin2: ", arg

class Mixed(Mixin1, Mixin2):
  def do_foo(self, arg):
    super(Mixed, self).do_foo(arg)
    print "I'm Mixed: ", arg

m = Mixed()
m.do_foo(1)

Fails because eventually super() finds a class that doesn't have a
do_foo attribute.

Replacing the above with:

class Mixers(object):
  def do_foo(self, arg):
    print "Mixers doesn't call super()"

class Mixin1(Mixers):
  def do_foo(self, arg):
    super(Mixin1, self).do_foo(arg)
    print "I'm Mixin1: ", arg

class Mixin2(Mixers):
  def do_foo(self, arg):
    super(Mixin2, self).do_foo(arg)
    print "I'm Mixin2: ", arg

class Mixed(Mixin1, Mixin2):
  def do_foo(self, arg):
    super(Mixed, self).do_foo(arg)
    print "I'm Mixed: ", arg

Works and gives:
>>> m.do_foo(1)
Mixers doesn't call super()
I'm Mixin2:  1
I'm Mixin1:  1
I'm Mixed:  1

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkr11qsACgkQJdeBCYSNAAMb4QCeK5LNFe1xGOql+9uzsfnAysdf
VqMAoI0n+TocTYZF7PIBjGyFmjXxX/Ja
=U8CQ
-----END PGP SIGNATURE-----