• Multiple inheritance and a broken super() chain

    From =?UTF-8?B?UGV0ZXIgU2zDrcW+aWs=?=@21:1/5 to All on Mon Jul 3 19:38:08 2023
    Hello.

    The legacy code I'm working with uses a classic diamond inheritance. Let me call the classes *Top*, *Left*, *Right*, and *Bottom*.
    This is a trivial textbook example. The classes were written in the
    pre-super() era, so all of them initialized their parents and Bottom initialized both Left and Right in this order.

    The result was expected: *Top* was initialized twice:

    Top.__init__() Left.__init__() Top.__init__() Right.__init__() Bottom.__init__()

    Now I replaced all parent init calls with *super()*. After this, Top was initialized only once.

    Top.__init__() Right.__init__() Left.__init__() Bottom.__init__()

    But at this point, I freaked out. The code is complex and I don't have the
    time to examine its inner workings. And before, everything worked correctly even though Top was initialized twice. So I decided to break the superclass chain and use super() only in classes inheriting from a single parent. My intent was to keep the original behavior but use super() where possible to
    make the code more readable.

    class Top:
    def __init__(self):
    print("Top.__init__()")

    class Left(Top):
    def __init__(self):
    super().__init__()
    print("Left.__init__()")

    class Right(Top):
    def __init__(self):
    super().__init__()
    print("Right.__init__()")

    class Bottom(Left, Right):
    def __init__(self):
    Left.__init__(self) # Here I'm calling both parents manually Right.__init__(self)
    print("Bottom.__init__()")

    b = Bottom()


    The result has surprised me:

    Top.__init__() Right.__init__() Left.__init__() Top.__init__()
    Right.__init__() Bottom.__init__()

    Now, as I see it, from the super()'s point of view, there are two
    inheritance chains, one starting at Left and the other at Right. But *Right.__init__()* is called twice. What's going on here?

    Thanks,
    Peter

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Damon@21:1/5 to All on Mon Jul 3 14:01:19 2023
    On 7/3/23 1:38 PM, Peter Slížik via Python-list wrote:
    Hello.

    The legacy code I'm working with uses a classic diamond inheritance. Let me call the classes *Top*, *Left*, *Right*, and *Bottom*.
    This is a trivial textbook example. The classes were written in the pre-super() era, so all of them initialized their parents and Bottom initialized both Left and Right in this order.

    The result was expected: *Top* was initialized twice:

    Top.__init__() Left.__init__() Top.__init__() Right.__init__() Bottom.__init__()

    Now I replaced all parent init calls with *super()*. After this, Top was initialized only once.

    Top.__init__() Right.__init__() Left.__init__() Bottom.__init__()

    But at this point, I freaked out. The code is complex and I don't have the time to examine its inner workings. And before, everything worked correctly even though Top was initialized twice. So I decided to break the superclass chain and use super() only in classes inheriting from a single parent. My intent was to keep the original behavior but use super() where possible to make the code more readable.

    class Top:
    def __init__(self):
    print("Top.__init__()")

    class Left(Top):
    def __init__(self):
    super().__init__()
    print("Left.__init__()")

    class Right(Top):
    def __init__(self):
    super().__init__()
    print("Right.__init__()")

    class Bottom(Left, Right):
    def __init__(self):
    Left.__init__(self) # Here I'm calling both parents manually Right.__init__(self)
    print("Bottom.__init__()")

    b = Bottom()


    The result has surprised me:

    Top.__init__() Right.__init__() Left.__init__() Top.__init__() Right.__init__() Bottom.__init__()

    Now, as I see it, from the super()'s point of view, there are two
    inheritance chains, one starting at Left and the other at Right. But *Right.__init__()* is called twice. What's going on here?

    Thanks,
    Peter

    Because the MRO from Bottom is [Bottom, Left, Right, Top] so super() in
    Left is Right. It doesn't go to Top as the MRO knows that Right should
    go to Top, so Left needs to go to Right to init everything, and then
    Bottom messes things up by calling Right again.

    --
    Richard Damon

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mats Wichmann@21:1/5 to Richard Damon via Python-list on Mon Jul 3 12:13:37 2023
    On 7/3/23 12:01, Richard Damon via Python-list wrote:
    On 7/3/23 1:38 PM, Peter Slížik via Python-list wrote:
    Hello.

    The legacy code I'm working with uses a classic diamond inheritance.
    Let me
    call the classes *Top*, *Left*, *Right*, and *Bottom*.
    This is a trivial textbook example. The classes were written in the
    pre-super() era, so all of them initialized their parents and Bottom
    initialized both Left and Right in this order.

    The result was expected: *Top* was initialized twice:

    Top.__init__() Left.__init__() Top.__init__() Right.__init__()
    Bottom.__init__()

    Now I replaced all parent init calls with *super()*. After this, Top was
    initialized only once.

    Top.__init__() Right.__init__() Left.__init__() Bottom.__init__()

    But at this point, I freaked out. The code is complex and I don't have
    the
    time to examine its inner workings. And before, everything worked
    correctly
    even though Top was initialized twice. So I decided to break the
    superclass
    chain and use super() only in classes inheriting from a single parent. My
    intent was to keep the original behavior but use super() where
    possible to
    make the code more readable.

    class Top:
    def __init__(self):
    print("Top.__init__()")

    class Left(Top):
    def __init__(self):
    super().__init__()
    print("Left.__init__()")

    class Right(Top):
    def __init__(self):
    super().__init__()
    print("Right.__init__()")

    class Bottom(Left, Right):
    def __init__(self):
    Left.__init__(self) # Here I'm calling both parents manually
    Right.__init__(self)
    print("Bottom.__init__()")

    b = Bottom()


    The result has surprised me:

    Top.__init__() Right.__init__() Left.__init__() Top.__init__()
    Right.__init__() Bottom.__init__()

    Now, as I see it, from the super()'s point of view, there are two
    inheritance chains, one starting at Left and the other at Right. But
    *Right.__init__()* is called twice. What's going on here?

    Thanks,
    Peter

    Because the MRO from Bottom is [Bottom, Left, Right, Top] so super() in
    Left is Right. It doesn't go to Top as the MRO knows that Right should
    go to Top, so Left needs to go to Right to init everything, and then
    Bottom messes things up by calling Right again.

    And you can see this a little better in your toy example by using begin
    *and* end prints in your initializers.

    Also, you might find that because of the MRO, super() in your Bottom
    class would actually give you what you want.

    And if not sure, print out Bottom.__mro__

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From risky sibam@21:1/5 to All on Mon Jul 3 11:16:18 2023
    https://guru.unika.ac.id/slot4d/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mats Wichmann@21:1/5 to Mats Wichmann via Python-list on Mon Jul 3 13:06:03 2023
    On 7/3/23 12:13, Mats Wichmann via Python-list wrote:

    To natter on a bit, and possibly muddy the waters even further...

    Now, as I see it, from the super()'s point of view, there are two
    inheritance chains, one starting at Left and the other at Right. But *Right.__init__()* is called twice.
    No: each class has just a single inheritance chain, built up when the
    class object is constructed, to avoid going insane. Yes, the chain for
    Left and for Right are different, but you're not instantiating *either*
    of those classes when you make a Bottom, so they don't matter here.
    You're just filling out a Bottom: it looks for init, finds it, and so
    would stop hunting - but then the super() call there sends it to the
    next class object in the chain to look for the method you told it to
    (also init), where it would stop since it found it, except you sent it
    on with another super(), and so on. Python is a bit... different :)
    (compared to other languages with class definitions)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Angelico@21:1/5 to [email protected] on Tue Jul 4 04:39:57 2023
    On Tue, 4 Jul 2023 at 03:39, Peter Slížik via Python-list <[email protected]> wrote:

    Hello.

    The legacy code I'm working with uses a classic diamond inheritance. Let me call the classes *Top*, *Left*, *Right*, and *Bottom*.
    This is a trivial textbook example. The classes were written in the pre-super() era, so all of them initialized their parents and Bottom initialized both Left and Right in this order.

    The result was expected: *Top* was initialized twice:

    Top.__init__() Left.__init__() Top.__init__() Right.__init__() Bottom.__init__()

    What happens when Top is initialized twice? This seems like a problem
    waiting to happen, and when you moved to using super(), you more than
    likely simplified things and fixed things.

    There are not two instances of Top to be initialized, only one.

    ChrisA

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Angelico@21:1/5 to [email protected] on Tue Jul 4 22:08:58 2023
    On Tue, 4 Jul 2023 at 22:06, Peter Slížik via Python-list <[email protected]> wrote:


    Also, you might find that because of the MRO, super() in your Bottom
    class would actually give you what you want.


    I knew this, but I wanted to save myself some refactoring, as the legacy
    code used different signatures for Left.__init__() and Right.__init__().

    This sounds like a mess that I would not touch. Unless something needs
    to be fixed, I wouldn't switch these to use super() - I'd leave them
    using explicit parent calls.

    However, I would acknowledge somewhere that this will cause
    Top.__init__() to be called twice on the same object.

    ChrisA

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?B?UGV0ZXIgU2zDrcW+aWs=?=@21:1/5 to All on Tue Jul 4 14:04:22 2023

    Also, you might find that because of the MRO, super() in your Bottom
    class would actually give you what you want.


    I knew this, but I wanted to save myself some refactoring, as the legacy
    code used different signatures for Left.__init__() and Right.__init__().

    I realized the formatting of my code examples was completely removed; sorry
    for that.

    Best regards,
    Peter

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alan Gauld@21:1/5 to Chris Angelico via Python-list on Tue Jul 4 23:33:51 2023
    On 03/07/2023 19:39, Chris Angelico via Python-list wrote:
    On Tue, 4 Jul 2023 at 03:39, Peter Slížik via Python-list
    The legacy code I'm working with uses a classic diamond inheritance.

    What happens when Top is initialized twice? This seems like a problem
    waiting to happen, and when you moved to using super(), you more than
    likely simplified things and fixed things.

    Slightly off topic but I wonder how many real world problems
    people have experienced having the top of a diamond initialized
    twice? The reason I ask is that I ran a maintenance team for
    about 5 years (early 1990s) working on various OOP projects using MI;
    in Lisp Flavors, C++(*) and a homebrew variant of C that supported MI.
    In that time I don't recall ever having problems with top objects
    being initialized twice (apart from redundant execution of code
    of course).

    In most cases the top object was so abstract that its init()/constructor
    was only doing generic type stuff or opening database sessions/networks
    etc which got lost and tidied up by garbage collectors.

    So I'm curious about how big this "big problem with MI" is in
    practice. I'm sure there are scenarios where it has bitten folks
    but it never (or very rarely) occurred in our projects. (Of
    course, being maintenance programmers, the problems may have
    been ironed out before the code ever reached us! But that
    wasn't usually the case...)

    (*) C++ is the odd one out because it doesn't have GC, but then
    neither does it have an Object superclass so very often MI in C++
    does not involve creating diamonds! And especially if the MI
    style is mixin based.

    --
    Alan G
    Author of the Learn to Program web site
    http://www.alan-g.me.uk/
    http://www.amazon.com/author/alan_gauld
    Follow my photo-blog on Flickr at:
    http://www.flickr.com/photos/alangauldphotos

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Angelico@21:1/5 to [email protected] on Wed Jul 5 10:27:37 2023
    On Wed, 5 Jul 2023 at 08:35, Alan Gauld via Python-list <[email protected]> wrote:

    On 03/07/2023 19:39, Chris Angelico via Python-list wrote:
    On Tue, 4 Jul 2023 at 03:39, Peter Slížik via Python-list
    The legacy code I'm working with uses a classic diamond inheritance.

    What happens when Top is initialized twice? This seems like a problem waiting to happen, and when you moved to using super(), you more than likely simplified things and fixed things.

    Slightly off topic but I wonder how many real world problems
    people have experienced having the top of a diamond initialized
    twice? The reason I ask is that I ran a maintenance team for
    about 5 years (early 1990s) working on various OOP projects using MI;
    in Lisp Flavors, C++(*) and a homebrew variant of C that supported MI.
    In that time I don't recall ever having problems with top objects
    being initialized twice (apart from redundant execution of code
    of course).

    It's important to distinguish between diamond inheritance and what the
    OP seemed to expect, which was independent hierarchies. (In C++ terms,
    that's virtual inheritance and .... the unnamed default type of MI.
    Non-virtual inheritance??) With independent hierarchies, the object is
    composed of two subobjects, each with its own regular
    single-inheritance tree, and unless you need to call a method on the
    duplicated grandparent from the second parent (in which case you have
    to cast before calling), it's perfectly natural to treat them
    separately. But with virtual inheritance - as is always the case in
    Python - there is only one object.

    Whether it's a problem depends entirely on what the initializer does.
    If it's idempotent and doesn't depend on any arguments that it didn't
    already get, we're fine! But if it does something like this:

    class Example:
    def __init__(self):
    self.button = Some_GUI_Library.Button("Example")
    self.button.add_to_window()

    then calling init twice will create a second button.

    It's easy enough to design a specification that is safe against double initialization; but then, it's also not that hard to design a
    specification that's safe against super().__init__() and odd
    hierarchies.

    Since the OP didn't show us any of the code, I had to mention the
    possibility here. Of course, it's entirely possible that it isn't
    actually a problem.

    In most cases the top object was so abstract that its init()/constructor
    was only doing generic type stuff or opening database sessions/networks
    etc which got lost and tidied up by garbage collectors.

    Remember though that diamond inheritance doesn't always happen at the
    top-level object.

    So I'm curious about how big this "big problem with MI" is in
    practice. I'm sure there are scenarios where it has bitten folks
    but it never (or very rarely) occurred in our projects. (Of
    course, being maintenance programmers, the problems may have
    been ironed out before the code ever reached us! But that
    wasn't usually the case...)

    Who said it's a big problem with MI? Diamond inheritance is in general
    a problem to be solved, but calling __init__ twice is a separate
    concern and usually only an issue when you try to treat MRO-based MI
    as if it were composition-based MI.

    ChrisA

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Ewing@21:1/5 to Alan Gauld on Wed Jul 5 12:26:35 2023
    On 5/07/23 10:33 am, Alan Gauld wrote:
    (*) C++ is the odd one out because it doesn't have GC, but then
    neither does it have an Object superclass so very often MI in C++
    does not involve creating diamonds! And especially if the MI
    style is mixin based.

    Even if all your mixins have empty constructors, in C++ there
    is still a diamond problem if they have any data members, because
    you end up with multiple copies of them.

    But C++ has the concept of virtual base classes, which avoids the
    diamond problem, albeit at the expense of making you explicitly
    call all the base class constructors in your ancestry.

    --
    Greg

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Angelico@21:1/5 to [email protected] on Wed Jul 5 10:51:46 2023
    On Wed, 5 Jul 2023 at 10:31, Greg Ewing via Python-list <[email protected]> wrote:

    On 5/07/23 10:33 am, Alan Gauld wrote:
    (*) C++ is the odd one out because it doesn't have GC, but then
    neither does it have an Object superclass so very often MI in C++
    does not involve creating diamonds! And especially if the MI
    style is mixin based.

    Even if all your mixins have empty constructors, in C++ there
    is still a diamond problem if they have any data members, because
    you end up with multiple copies of them.

    But C++ has the concept of virtual base classes, which avoids the
    diamond problem, albeit at the expense of making you explicitly
    call all the base class constructors in your ancestry.

    Yeah, non-virtual MI in C++ is basically composition with a shorthand
    for calling non-conflicting methods or accessing non-conflicting data
    members. Point of random interest: Pike actually allows that sort of "composition MI" but will give you back an array of all parents when
    you seek a superclass's method, giving a very elegant syntax for MI.

    inherit Thing1;
    inherit Thing2;

    void method() {
    ::method(); //this is actually calling an array of two methods
    }

    Python's way of doing it requires that every class choose to cooperate
    in the MI and then be aware that they are all operating on the same
    object. Pike's and C++'s can sometimes be used as composition in
    disguise, but in general, MI does require proper cooperation.

    ChrisA

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alan Gauld@21:1/5 to Chris Angelico via Python-list on Wed Jul 5 18:40:39 2023
    On 05/07/2023 01:27, Chris Angelico via Python-list wrote:

    So I'm curious about how big this "big problem with MI" is in

    Who said it's a big problem with MI?

    I think it's a very common perception, particularly with
    newer programmers who have never used it in anger. Any
    time anyone discusses MI it seems somebody will jump in
    and warn about diamonds etc. As a result many steer clear
    of MI, which is a shame.

    My personal experience of MI is that used appropriately
    it is a powerful and useful tool. But it must be used
    in a true is-a type relationship and not just as a kind
    of cheap reuse mechanism - that's when problems start.

    Also, mixin style MI is particularly powerful but the
    protocol between mixin and "subclass" needs to be carefully
    designed and documented. Like any powerful tool you need
    to understand the costs of use as well as the potential
    benefits.

    --
    Alan G
    Author of the Learn to Program web site
    http://www.alan-g.me.uk/
    http://www.amazon.com/author/alan_gauld
    Follow my photo-blog on Flickr at:
    http://www.flickr.com/photos/alangauldphotos

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)