Conf42 Python 2021 - Online

Whose Method Is It Anyway?

Video size:

Abstract

The very thought of multiple inheritance inspires fear and loathing in many programming languages, but not in Python! In this emoji-powered talk, you’ll learn how Python uses the method resolution order to handle multiple inheritance, and how to put this to work for you.

Python does a shockingly good job at handling multiple inheritance. In this emoji-powered talk, learn how Python figures out what method to call in a multiple inheritance situation. Armed with this knowledge, you’ll be prepared to swing in as the hero the next time your team is scratching their head and asking “why’s THAT code getting run??”

In the second half of this talk, you’ll learn how to actually USE multiple inheritance, in the form of mixins, to make your code easier to maintain.

Summary

  • How does Python handle multiple inheritance? It's not as scary as it sounds. The explanation is coming out of my book, dead simple Python. This knowledge can be really helpful for writing Python code as Python code.
  • A calzone inherits from both pizza and sandwich. We need to merge in the superclass linearizations of both of those objects. The order that we inherit from is very important. It can clear up a lot of surprises regarding which method is being called in multiple inheritance.
  • Multiple inheritance can be tricky, but there's a simple solution. The order of inheritance matters. Instead of inheriting from sandwich and then pizza, we can inherit from pizza and then sandwich on the pizza sandwich class. This can come in handy when dealing with complex multiple inheritance.
  • This is where mixins come in handy. A mix in is a class that contributes methods and or attributes. It can rely on the attributes and methods of the class that's using the mix in. Let's do a little bit of live coding here. Use click to write a command line interface.
  • A coffee shop is going to want to know, know what drink you want first. And then they'll ask you about what you want to eat. Once again, we're using this brew and cook methods. Where are these going to come from?
  • You should be documenting everything, really, but especially mix ins because it's both going to provide methods and attributes potentially, but it also has expectations. Get in the habit of documenting your mix ins.
  • A short order cook mix can be imported from restaurant employees. The mix has to have a self dot ready attribute on the class using it. Then we need to actually brew the particular order for the customer. This can be done in a virtual environment.
  • Short order cook mix in and barista mix in don't inherit from each other. Solution is to explicitly call brew on the method that we want. Multiple inheritance and abstract based classes get really kind of interesting.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
It? Whose method is it anyway? Anyway, that's the question that scares a lot of developers when we talk about multiple inheritance. If you have a class, say a class c, that inherits from classes a and b, then if classes a and both implement a function, say foo, because I have no imagination here. Then if you call c foo, which version of foo gets called the one on a or the one on b? That's the trick. It gets more complicated if then you have a shared parent class, like just say alpha. And so A and B both inherit from Alpha. What if that also defines foo? And so that gets very complicated, very difficult. We call that the diamond inheritance pattern, or the deadly diamond of death if you're coming from the Java world. And it generally just scares the living daylights out of most developers, especially those who design languages. So they just don't support multiple inheritance. Simple Python, however, is not as afraid of this. It's quite call right with working with multiple inheritance. It has some pretty cool patterns for solving it. And that's what I'm going to be exploring in this talk. How does Python handle multiple inheritance? It's not as scary as it sounds. So the explanation I'm going to be giving is coming out of my book, dead simple Python, which is coming up from no search press, hopefully this year. Writing a book, getting a book edited takes some time. That's why if anyone's been following this, the date keeps being pushed out a little bit. But it's just edits. It is coming anyhow. So dead simple Python, I explore call the idiomatic patterns of the Python language why we do things a certain way, why we call certain things pythonic. And this knowledge can be really helpful for writing Python code as Python code. So whether you've been working in the language for a year or two days or a decade, there's probably something here that is going to be insightful. So the way Python handles this multiple inheritance situation is through what is called the c three method resolution order, or c three mro, which frankly sounds like a droid out of Star wars. Now, the whole point of the C three MRO is to determine the superclass linearization of a class, which is just a big fancy term that you can use to impress people at cocktail parties. So the superclass linearization is determined like this. The C three MRO first looks at the class that we're figuring out the linearization for. That's going to be the first item in the linearization. So in this case, food. Food inherits from object. So the next thing that goes in the linearization is the thing it inherits from object. Okay, that's pretty easy. Now, this is important because Python uses this superclass linearization to figure out where a method comes from. Ergo, method, resolution, order. So let's say we want to call the eat function on food. We'll call food eat. And Python says, where's the eat function? It's going to go through the superclass linearization left to right. Is it on food? No. Let's check object. If it's still not there, this is when it would say it can't find the method. As soon as it encounters that method in any of these classes, it stops looking and it's good. We'll see that again in a moment. Let's consider a class pizza. Pizza inherits from food. So once again we start the superclass linearization with pizza. Now we need to merge in the superclass linearization for food, which is food and object. We just saw that in the last step. There's two parts to this list that we are merging in. The head is food. It's the first item in the list we're merging. The tail is everything that comes after it. In this case, object. So we're going to look at the head of the list we're merging, which is food, and we can bring that in because there's a simple rule. If the head does not appear in any tail, we can bring it in. That'll make more sense as we go on. Now, once we bring it in, we remove food from the list to merge. The new head is object. So we can bring that in as well, because it doesn't appear in any tail. There is no tail. So the superclass Linearization for pizza is pizza, food at object. Let's look at a sandwich. Sandwich is also food. Same sort of thing going on. We need to merge in the superclass linearization for food. So we start out with sandwich, and then we look at the head of the list we're merging in, which is food. We can bring that in, and then we remove it from the linearization to merge. And the new head is object, which we can also bring in. The superclass linearization of sandwich is sandwich, food and object. It's pretty simple so far. Nothing really surprising. Doesn't make a very good talk if I were to end here. So let's make it more complicated. How about a calzone? Thank you to Kevin McCauley for allowing me to use this. This comes from his Seinfeld 2000 emoji set. So check him out. So a calzone inherits from both pizza and sandwich. So we need to merge in the superclass linearizations of both of those objects. Now we're going to work from left to right. The order that we inherit from is very important, and you're going to see this more and more as we go on. Now, in these two linearizations, we're merging in. We have two heads, pizza on the left hand, linearization and sandwich on the right hand. The rest is just in the tail. So the tail for the pizza linearization is food and object, and the tail in the sandwich linearization is food and object. Okay, so we start out with calzone in our new linearization, and we're going to look at the leftmost head, which is pizza. Now we can bring pizza in because it does not appear in the tail of any linearization here. No other instances of pizza. So we're good. Once we bring pizza in, we remove it from the linearizations to merge. We now have a new head. So the leftmost head is food. Well, we can't bring that in because it appears in the tail of the other linearization, the one belonging to sandwich. No go. So now we look at the next leftmost head, which would be sandwich. We can bring sandwich in because it doesn't appear in any tails. Okay, when we bring this in, we're going to go back to the leftmost head again, which is food. Now we can bring it in. Why? Because it is the head in both places. It appears it's the head in what remains of the linearization for pizza. And it's the head in what remains of the linearization for sandwich. And it's not in any tails. So we can bring that in. The new head in both is object. We bring that in as well. So the superclass linearization of calzone is calzone, pizza, sandwich, food, object. So if we want to call the eat method here, Python asks, well, where is the eat method? We're going to use the superclass linearization. We first look at calzone, and if we don't find it there, then we look at pizza, and if we find it there, we're done, we're good. We call eat on the pizza class and move on. We will never get to the sandwich eat method or the one on food. And this is where it becomes so helpful to understand how this method resolution order works, because it can clear up a lot of surprises regarding which method is being called in multiple inheritance. So we check in order until we find it. Okay, let's look at a pizza sandwich. You know, what a pizza sandwich is. You wake up, you're hungry, and pizza the night before, you grab two slices, stick something in between maybe, and just put it together and eat it cold. This is like the breakfast of champions here. Okay, so a pizza sandwich inherits from sandwich and pizza. So basically the same sort of scenario from before. So we start a new linearization with pizza sandwich, and then we're going to look at the lists we need to merge in, or the linearizations we need to merge in. So the heads here are sandwich and pizza respectively. So let's look at the leftmost head, which is sandwich, doesn't appear in any tails. We can bring that in the new head there. The new leftmost head is food. Can't bring that in because it's the tail. And the other linearization, no go. So then we look at the next head, which is pizza. We can bring that one in because it doesn't appear in any tails. We go back to the leftmost head, which is still food. Now we can bring it in because it's the head in both places, it's not appearing in any tails. Super good. New leftmost head is object. We can bring that in because it's ahead in both places. Superclass linearization of pizza sandwich is pizza sandwich sandwich, pizza food object. Cool. Now let's get crazy. Let's have a calzone pizza sandwich, which really needs to be a thing. So we start out with calzone pizza sandwich, and we're going to need to merge in the superclass linearizations for both calzone and pizza sandwich. Hang on to your hats. What are our heads here? We have calzone and we have pizza sandwich. Okay, so leftmost head, calzone. It's the only place that shows up. We can bring that in. The new head is pizza. Can't do that because it's in the tail of the linearization for pizza sandwich. Okay, so let's look at the next leftmost head. Well, that would be pizza sandwich. We can bring that in. Back to the beginning. Pizza. Still can't bring it in. How about sandwich? That's the head in the second list. Can't bring that in because it is in the tail of the first list. Now we're at an impasse because we can't bring in pizza and we can't bring in sandwich. And at this point the c three mro just blows up and we get type error. Cannot create a consistent method resolution. And this is why multiple inheritance scares everybody. But there's thankfully a fairly simple solution in this case. Remember how I said that the order of inheritance matters. Since we know that we need to be using our calzone and our pizza sandwich classes together, we can change the order that we inherit from on pizza sandwich. So instead of inheriting from sandwich and then pizza, we can inherit from pizza and then sandwich on the pizza sandwich class, just like we do with calzone. That is going to effectively swap these two classes in the linearization. So we have pizza and sandwich in both cases. Now this is going to work a lot smoother, so let's try it out. So check calzone first. We can bring that in. Great. New head. Pizza. Can't do that. Next. Pizza sandwich. Cool. Back to the beginning. Pizza head. In both places we can bring it in. Sandwich. Cool. And food and object. And booyah. So the superclass linearization for calzone pizza sandwich is now calzone, pizza sandwich, calzone pizza sandwich pizza sandwich, food, and object. Now, a little side note here is that if you're dealing with very complex multiple inheritance, you can't necessarily just go running around rearranging things hoping it's going to work. You're going to need to figure this out. There are some other patterns, which I'm not going to go in, where you can use another class that just inherits from other classes. It doesn't have any contents, and you can kind of use that to get around some problems with multiple inheritance. Raymond Hediger has a really good article on this called super, considered super. So check that out if you want to know more about some of this advanced stuff. But for most uses, this should do. Now another little note here is what if we want to explicitly call the eat method on, say, pizza instead of calling it on calzone? Well, we can do that like this. So on calzone pizza sandwich, I define an eat function, and the only line of code I need in my eat function is pizza eat, and then I pass the self argument explicitly. So I'm just calling the method I want on the class. I want it to come from class being something in that superclass linearization. And I have to pass self explicitly because of course the only time self is passed implicitly is if you're calling on an object. We're not calling on an object, we're calling on a class. So we have to pass self so that we know what instance the method is being called on. So we just call pizza, eat and pass self. So where is this even useful? Well, I think one major place where this can come in handy is this entire concept of mixins, which is applied multiple inheritance, and mixins are one of those things you really wind up missing when you leave Python and go into a no multiple inheritance type language. So let's consider a diner. A diner is kind of a neat place to hang out. You can order food. You can find coffee sitting in the percolator, probably hour old, so it's a bit stale. But hey, it's coffee. Coffee is coffee when you're desperate. And then you have a coffee shop. Coffee shop. You can get your fancy coffee. You can get your french press or your caramel macchiato. That's my thing. Or whatever you like. And they might have some food there too, but that's not their main gig. Their main thing is coffee. Now, you could say these are both classes. More importantly, they're classes that don't inherit from one another, because that doesn't make any sense. A coffee shop is not a type of diner, and a diner is not a type of type of coffee shop. Okay, they might both inherit from restaurant, but they're going to each have some functionality that is unique to them, and it doesn't really make sense to figure out where they're going to share functionality from that parent class. It doesn't make a lot of sense because of their differences. This is where mixins come in handy. So we might have a short order cook working at the diner. He's the one responsible for making all of that fancy food. Short order cook needs restaurant data to be able to work. A cook can't really cook without having access to the recipes. And of course, the customer orders and the special of the day, the stuff that the restaurant provides that he could not possibly know by himself. And the short order cook produces food for the restaurant. It really doesn't make a lot of sense to have a short order cook without a restaurant. But there he is, happily working in that restaurant. And then a coffee shop has a barista. The barista also needs restaurant data, recipes, customer orders, specials, same sort of deal. Once again, the barista produces drinks, but produces drinks specifically for the restaurant. And as with the short order cook, it doesn't make a lot of sense to have a barista that is completely separated from a coffee shop in some fashion. Now, maybe that coffee shop decides after a while they want to offer some breakfast food, and so they also hire a short order cook. And she can do a lot of the same stuff that the short order cook in the diner does, okay? Actually, she can do all of the same things that the cook in the diner can. And it would be helpful to be able to maintain one copy of the short order cook code shared between the two instances. This is why we have mixins. So a mix in is a class that contributes methods and or attributes, and it can rely on the attributes and methods of the class that's using the mix in the shorter to cook and rely on the orders of the customers. But a mix in is not intended to be to stand alone. In fact, in many cases you can't even instantiate a mix in by itself. It's just not intended to be used alone. It's only intended to be uses by another class. So youll could say that a mix in is technically a form of composition. It's just a form of composition that happens to use inheritance as the means of composing. And if that doesn't completely blow your mind, then you are a much more savvy code than I. So this is quite abstract. So let's do something real with this call. We let's do a little bit of live coding here. So I need to start with, I have my repository set up here, and this is just your standard repository set up source structure. Okay. And I'm going to create a new file in here and this is just going to be my main. Let's start with main. That can be helpful. So we know where we're going. Get this terminal out of the way for now and get this out of the way. All right, so let's just stick the whole shebang in here and then I'm going to use a library called click. If you haven't used click, it's really an easy way to write a command line interface. It's just going to save us a lot of time. Definitely check it out if you do anything on the command line with Python. So I'm going to wind up writing a restaurants class or restaurants module and it doesn't exist yet, but I'm just going to put it in here because I know it's going to and I'm going to have diner and a coffee shop as classes defined in restaurants. Let's set up a couple of commands. So I would like to be able to just have a command called coffee shop. And let's instantiate a coffee shop here. Java was one of my favorite coffee shops here in town. They unfortunately closed because of the pandemic, but they were lovely, had some really good french press. Okay, so in a coffee shop you're going to take the order and then you're going to deliver the orders. I'm obviously oversimplifying it, but it should work. We also need a diner, same sort of deal here. So I have the owl cafe, which is the other place I used to hang out, also closed. Now I don't have any hangouts now. I'm very sad. Okay, so in a diner as well, you take the order and then let's just print a new line here and then deliver the orders. Okay, so I'm not going to explain this part much. If you want to know what I'm doing here with click, you can just look it up on their documentation. Pretty easy to use. Add these commands in here's. Okay. And then your usual invocation. Okay, nothing surprising there. But the basic idea is that we need to have two different classes here and they can both take orders and then deliver orders. Okay, that's pretty straightforward. So let's make that restaurants file do. All right, so I'm going to, I'll do that a bit. Let's start by actually writing this out. So have a diner here and we need to initialize it with the name of the establishment, whatever that happens to be. And I want to have a default dictionary, basically just a dictionary of orders. So using the name of the customer as the key and then a list of orders. I'm using a default dict here so that if the customer is not already in the dictionary, then we can just append anyway. And it's going to start us out with an empty dictionary or an empty list. Okay, so we saw that we have two methods here that we need. So first we're going to greet, get that out of my way. So we're going to greet the customer and then call lock it turned on. And then we're going to accept some input, namely the customer's name. This is what we'll use for storing the order. And then if anyone wants to say, hey, he's being stereotypical. No, the waitress at the owl cafe was very much what you find into the movies. So she was fantastic. And then of course, the important question, do you want coffee? Which my answer is always, yes. But as you know, in a coffee shop you can get anything you like, but in a diner you just get the one thing you can't get, the fancy stuff. If you ask for a cappuccino or caramel macchiato or what have you, they're just going to kind of give youll a blink, look and pour the stuff out of the pod that they have now they can do the cooking and that's great. They'll do some awesome food, but they're not going to handle anything complicated with the coffee. Okay. And then we have our delivers order, deliver orders here. So for each of the items in self dot ready. So all of the stuff that's ready, I'll just call out here, here's your order. Let's just put that in lowercase order, lower name. Okay, so there's that. So we're going to take the order from the customer and we're going to deliver the orders. Now, you note I used a couple of functions I haven't defined yet. I have this brew function for the coffee and I have this cook function and I'm going to come back around to those. So I want to get these classes written first. So now let's write the coffee shop and see what difference is emerging. I don't need an empty list. What's wrong with me? Okay, so I have my coffee shop here and I have my initializer. Actually, I'm going to go ahead and just steal this initializer because this one is exactly the same. And I know I could have created a restaurant class, but I'm trying to save time here. Actually, we use the same welcome statement, too. So we have a little more typing. So now we need to get the name of the customer as before it call. Right. And then a coffee shop, unlike a diner, is going to want to know, know what drink you want first. That's the first thing they're going to want to know. And then they'll ask you about what you want to eat. Now, yes, I know this user interfaces aren't really very well thought, but because you can't brew scrambled eggs, well, you can, but it's going to be a right royal mess. But I think you can forgive the terrible example. Okay, so we're going to brew the coffee and we're going to cook the food. All right. Now I'm going to deliver the orders and actually I can hijack this part as well again, because that's not changing. And then they do deliver them a little differently in the coffee shop. They just kind of yell your name out for the entire world to hear, which is especially amusing when they get it wrong. I am one of those people that has a prosaically easy to pronounce name, at least in many english speaking countries, but I really feel for my friend Bojan. Okay, so we take the order, deliver the orders pretty straightforward. But once again, we're using this brew and cook methods. So where are these going to come from? Well, this is where we're going to get into these mix ins here. So I am going to create a couple of mix ins and I'm going to create them first and then I'm going to come back and modify this. So I'll just create a file called mixins Py. I could call it whatever I want. Actually, I suddenly changed my mind. I'm going to call it employees Py. Okay. Now, typically in python, we're going to append the word mix in to the name of our class just to make sure that we know that, okay, this is not a normal class. You can't just instantiate this. You also should get in the habit of. You should be documenting everything, really, but especially mix ins because it's both going to provide methods and attributes potentially, but it also has expectations. In this case, we need a self dot ready attribute, which is a dictionary of key is a string and then it's a list of strings for the values. So that is really helpful to document right up front. So get in the habit of documenting your mix ins. Document everything, really. Okay, so we're going to cook the order and then I'm going to do self ready key customer, and I'm going to append the order. Now, of course, my pylance is really not happy with me here because ready doesn't exist on this class. And I'm going to tell Pylance basically to eat rocks because in this case, no, the mix is not going to provide self ready. That is, again, coming from over here. So it doesn't have to have it here. Now, I also need to have a brew function because you can get that coffee in your diner. And I'm going to just borrow this again. So if they want that, just going to pour drip coffee for store, nothing fancy. And it doesn't matter what you put into this. I would like a tall decaf cappuccino. That's not going to make a lick of difference. You are still just going to get coffee. Nothing to it. Okay, so there's our short order to cook mix in. Now, how do we use this? Well, over in restaurants we are going to inherit and you'll see that I'm importing it from restaurant employees. So I have my mix in here. I inherit. But remember, this is composition, technically not inheritance, because a diner is not a type of short order cook. It has a short order cook. So that's the one wrinkle with mixins you kind of have to get in your head. So your diner has your sort of our cook. Now let's jump back over to employees here and let's set up our barista mix in. Same sort of thing here. Now, we're only providing the brew method with barista, and I'm going to borrow this again. Same sort of expectation has to have a self dot ready attribute on the class using it. So same call signature and the same logic for not doing anything. Then we need to actually brew the particular order for the customer and we're going to append the order. Now, the type ignore obviously won't work. If you're using flake gate, youll have to use a different suppression comment for that. Just a note. All right, so now we have our brew method for barista mix in. So I'm going to go over here to the coffee shop, and we're going to incorporate the barista mix in and make sure I import that here. Now, we're most of the way there, but there's one problem still, and that is we have no cook method. And. Oh, that's right. We need to have that short order cook. So let's put a short order cook mix in. Let's put her in there. So we now have a shorter to cook. Excellent. Okay, so this should work if all is typed correctly, big old scary should work thing. Okay, I don't see any errors from pylance, so let me just go into my virtual environment here. And because I have my setup py done, I can just install this as an editable in my virtual environment, which I already have activated here. Just give that a moment and it might have to reinstall. Click. Okay, there we go. So we have restaurant. So I can just invoke restaurant directly here. And I am going to start by calling diner. Okay. Welcome to the owl cafe. What's your name? Jason. What would you like, honey? You know what? I really like their hash browns. Okay. You want coffee? Yes. Okay, so here's your coffee and my hash browns. Now, if I do that again and I try to do my cappuccino, I'm still going to just get coffee and note I get the coffee first, and then I get the hash browns. So that works exactly as I expected for a diner. Now let's go over to the coffee shop. What's your name? My name is Jason. What can I get started for youll? I really want that cappuccino still, I think. And anything to know. Let's see how your hash browns taste, shall we? Okay. Well, hey, where's my cappuccino? That's not right. I just got normal coffee. What's going on? Well, remember our method resolution order. If we go over here to restaurants and we look at coffee shop we have short order cook mix in and barista mix in. What does that mean for our method resolution order? We start with the coffee shop and then we look left to right. Okay, so short order cook mix in followed by the barista mix in followed by object, because they don't really inherit from anything. Okay, well, crud, there's our problem. So when we call this brew method, it's going to go through this in order. Does it find it on coffee shop? No. So how about on short order cook mix in? Yes, as a matter of fact, it's right here. And this is why we're only getting coffee. So there's two ways to fix this problem. First way is to reverse the order of inheritance, which would work with just basic simple mixins. Just change the order of inheritance, barista first, followed by the short order cook. That should be fine, because barista does not define a cook method. So if we call for brew, then it's going to go through here and it's going to find brew there and then it's going to look for cook and it's going to find it there. So that's good. That should be fine. Let's try that out. I still want that cappuccino. Thank you. And hash browns. Now I get my cappuccino. So that is the solution there. But what if you're dealing with a more complicated situation? Perhaps you can't change that order of inheritance. Maybe you need that just to be able to get that consistent method, resolution order. You keep getting that error message. We talked, but earlier then in that case, the solution is to explicitly call brew on the method that we want or on the class that we want. So we have our brew method here in coffee shop and I'm just going to call baristamixin brew. I have to pass self explicitly because I'm calling on a class, not on an instance. And then I pass an order and customer. Now it's not going to matter as much. I can put in my name, I can ask for my cappuccino and I can get my hash browns. And it still works, even though short order cook mix in comes first. Why? It's because method resolution order, we look for brew under coffee shop first and we find it. This is called, and it explicitly calls brew on barista mix in. So we never even reach short order cook mix in on the method resolution order. We stop short here and explicitly call what we want. So that's a very basic idea of what you can do with mixins. And there are a lot more things you can do with them, they can be very powerful tool, obviously, as with anything disclaimer not everything that looks like just because you have a hammer does not mean everything is a nail. Not everything that looks like a nail is a nail. So make sure you consider if you have other options first, because mixins can also complicate things. Another side note is multiple inheritance and abstract based classes get really kind of interesting. So do be aware of that if you're kind of in some of the more complicated territory, especially if you touch metaclasses with a 50 foot pole. But besides that, if you have functionality that makes the most sense as a class, like a short order cook, like a barista, and you want to be able to share that between multiple classes, then a mix in can be a really great way of accomplishing that goal. So that is whose method it is anyway. So if you want more of more content like this, youll can check me out at codemouse 92. Com. You can also find me all around the Internet as codemouse 92. It's my ubiquitous username I post, but on Twitter I can also be found on the dev community and on Freenote IRC. And of course keep your eyes open for dead simple python coming out hopefully later this year from Nostarch press. Definitely will be out before we get too far into 2022, so keep your eyes out for that. And thank you very much for watching.
...

Jason C. McDonald

CEO @ MousePaw Media

Jason C. McDonald's LinkedIn account Jason C. McDonald's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways