Conf42 Python 2024 - Online

Functor Zoo

Video size:

Abstract

Leverage the abstractions and composability of functors. Do you want to use covariant or contravariant functors to lift regular Python functions and compose them without side effects? Interested in building highly extensible and composable data pipelines with pro-functors? This talk will show you how.

Summary

  • In this talk, we will look at functions and how to use them in Python programs. This talk requires no prior knowledge of functions or category theory. The talk is for Python programmers and is designed for ease of understanding.
  • To visualize the code, we can draw types as dots and functions as arrows. A diagram like the one we are seeing here represents a category in category theory. Let's move on to see how functions are related to categories.
  • In python, we can combine categories and functions to create new functions. With the I O function, we are able to keep the kernel of our code free of side effects. This solution can turn a partial function into a total function while retaining composability.
  • Next, let's turn our attention to a kind of functions called functors in Python. In python code, a functor class takes only one type argument, whereas a profunctors class takes two type arguments. A perhaps very helpful way for visualizing how aprofunctors works is through a diagram.
  • In summary, we've introduced the concept of categories. Then we covered variant contravariant closed applicative functions as well as functors and profunctors. Hope you all enjoyed the talk.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hi, thank you for coming to my talk. In this talk, we will look at functions and how to use them in Python programs. Don't worry if you haven't heard of functions before. This talk requires no prior knowledge of functions or category theory. The talk is for Python programmers and is designed for ease of understanding. There will be places where we will choose to simplify things at the expense of skipping some math details. The code examples are based on an open source Python package called funklift. The links to the GitHub repositories of funklift and its tutorials are shown on the slide. Before we move on to the meat of the talk, a quick introduction of myself. My name is Char Wu. I'm a software developer, grew up in Taiwan, and I'm based in the San Francisco Bay area for the past 20 years. Now, today's agenda is very simple. We will start with a motivating example and then we will introduce various functions with even more examples. Let's start with a simple function, get price that takes an item and returns the item's price. To visualize the code, we can draw types as dots and functions as arrows. As you can see, item is a dot in the diagram and int is also a dot. Get price is an arrow that goes from item to int. Now we have another function. It's pricey and we can draw it as an arrow as well. This time the arrow goes from int to bool. With those two functions, what interesting things can we do with them? Well, we can compose the two functions to form a new function. Notice that the composition of functions in code corresponds nicely to the composition of arrows in our diagram. A diagram like the one we are seeing here represents a category in category theory. Essentially, a category consists of dots and arrows, plus some properties about how arrows compose. We will call our category here the category of types. In this category, the dots are Python types and the arrows are Python functions. Now let's move on to see how functions are related to categories. Here we have a function is even that goes from into pool and we have a list of numbers. 1234 we want to map is even over the numbers. One way to achieve that is by using this comprehension. Another way is to use the CDS class of the funklift package. As you can see from the code here, the CDS class has a method called fmap. Fmap takes the function is even and maps it over the list of numbers behind the scenes. On the outside, we can think of fmap as lifting is even to a function that goes from c list of int to c list of bool. If we put the is even function and its lifted counterpart side by side, we can see that C list is actually a mapping from a source category to a target category. Such a mapping is called a functor if it satisfies certain properties. In our case, C list is a functor. The source category and the target category of C list are both the category of types. A functor is called an functor when its source and target categories are the same. Okay, now that we've introduced the concepts of categories and functions, let's see some functions in action, and let's also see what problems they solve. It's common to have code that performs some kind of input or output. For example, we might have code that sends a request over a network, reads a file, or writes a record to a database. Such I O actions are side effects that make our code harder to reason about because they break referential transparency. Here on the side, the function get number side effect on the left has side effects because it reads user input from a console. To avoid such side effect, we can wrap the I O action into an instance of the I O class. With that, when we call the get number function on the right, no actual reading from the console will take place. Rather, we simply get an I O object that represents the I O action. The code here shows what we can do with the I O object returned by the get number function, even though no actual I O has happened yet. When we call get number, that does not stop us from mapping the is even function over the nonexistent number. Eventually, when we are finally ready to incur the actual I o, we would do so by coding the unsafe run method on the I o object. Although our example here is simple already, it shows that with the I O function, we are able to keep the kernel of our code free of side effects and push the actual I o actions to the boundary of our program. Now let's look at another example here. At the top we have a function that takes an integer and returns ten modulo that integer. The function is a partial functions because it's not defined when the input integer is zero. To fix that, we can write the function at the bottom. The function at the bottom is now defined for all input values. However, we now have a different issue, and that is the new function is not very composable with other functions. Here's an illustration of the lack of composability. As you can see, the code on the right tries to compose the two functions on the left, because the ten mark by functions returns either an integer or none. It is not very composable and we have to use if else statements when we try to compose it with other functions. Is there a solution that can turn a partial function into a total function while retaining composability? Yes, that solution is the option functions. As the example here shows, the ten mark by function now returns an option of int. The option class of the funklift package has a subclass called nothing and another subclass called sum. Nothing represents the absence of a value. On the other hand, sum represents the existence of a value. By the magic of functions and their fmap methods, we can compose ten map by with other functions without any if else statements in the code. Just like how we visualize c list as a functor, we can visualize option as a functor. The functor maps dots and arrows from a source category to a target category. So far we've been composing functions. It turns out that we can compose functions too. This diagram shows that we can compose the option functor and the c list function. The result of the composition is a new functions, and we call that new functions c list after option. Like any other functions, the new functions is a mapping between two categories. It maps dots to dots and arrows to arrows. If we can compose functors the way we compose arrows, does that mean functors are arrows in some sort of category? Notice that in this diagram, each rectangle represents a category. If we collapse the rectangles into dots, we will get a category whose dots are smaller categories and whose arrows are functions. Here's what that category looks like in a diagram. We call it the category of small categories. Of course, we don't just look at theories, we want to see some code, right? Absolutely. The code example on this side shows how to compose functions in python. Here, the variable nums is a c list of option objects. If we call fmap on nums, we will not be able to map over the numbers inside the option objects. In order to do that, we need to compose the two functions first to form a new functions, and we do that by using the compose class of the funklift package. Once we have the composite functions, we can call fmap on it, just like we do with any other functions. This is really fantastic. In this diagram, capital f denotes a functions. The functions we've seen so far are called covariant functions. Here, contravariant basically means that the two vertical arrows in the diagram point in the same direction. In other words, when a functor maps a source arrow to a target arrow, if the mapping does not change the direction of the arrows, then the functor is covariant. On the other hand, if the two vertical arrows in the diagram point in the opposite direction, then the functions is contravariant. For covariant functions we have the fmap method. For contravariant functions we have the cmap method. Similar to fmap, the cmap method lifts a function from type a to type b to a function that goes from f of b to f of a. Here's an example of a contravariant function called predicate. A predicate is something that is either true or false. In our example, we first have a predicate that will be true if we give it an even integer, and as the diagram shows, we can use cmap to convert that predicate of int into a predicate of str. With the converted predicate we can pass it the number six as a string, and it will tell us if that's an even number or not. The next type of functions we will look at is called applicative functions. Before we introduce applicative functor, we need to first get to know a special kind of categories called closed category. In the diagram on the side, there are two categories. The category on the left has a dot for type a and a dot for type b. If the category also has a dot in gray for the functions type a to b, plus some other properties, then the category is called a closed category. An example of a function type is into bool, as shown in the blue box. Now let's turn our attention to the category on the right. The category has a dot for the type f of a and a dot for the type f of b. Because it's also a closed category, it has a dot for the function type f of a to f of b in green. Between the two categories we have a functions that maps dots to dots and arrows to arrows. In particular, the functor maps the gray dot a to b on the left to the blue dot, f of a to b on the right in blue. Now, it may be the case that the blue dot and the green dot are not related at all. However, if they happen to be one and the same dot, then we say that the functor preserves the structure of the source category, and we call the functor a strict closed functor. Requiring the blue dot and the green dot to be the same dot is a rather strict condition. A somewhat more relaxed condition is to not require the two dots to be the same. Instead, we merely require that there's an arrow between the two dots in such a case we call the functions a lex closed functions. Another name for another name for next for next closed functions is applicative functions, and we call the arrow between the blue dot and the green dot app. That's a lot of theory. Let's see an example in code. Here we have a curried function for summing two integers. Currying means if a function takes multiple arguments, then the current function will take one argument at a time. In this example, we start with sum of 20. Then we f map the current read function over it. What we end up with is an option of end to end, which is shown as the blue dot in the diagram. Because our option functions is an applicative function, we have the app arrow that takes us from the blue dot to the green dot. The green dot is essentially a function that takes an option of int and returns another option of int. So if we give that function sum of 30, we will get back sum of 50 as the result. Next, let's turn our attention to a kind of functions called functors in Python. If we have a type a and a type c, we can form the tuple type a comma c. Similarly, if we have a type b and a type d, we can form a tuple type b comma d. And if there's an arrow from a to b and another arrow from c to d, we can take those two arrows and form a tuple of arrows that goes from a comma c to b comma d. Now, if we have a functions that maps a comma c to f of a comma c and b comma d to f of b comma d, and the functor also maps the tuple of arrows to a lifted tuple of arrows, then we call such functor a functor. In python code, a functor class takes only one type argument, whereas a functor class takes two type arguments. Let's see an example of a functor. The functor in the example is the either class from the funk devt package. As its name suggests, either represents one of two possibilities. We represent those two possibilities with two classes left and right. In the coexample, we take two functions, add one and negate, and we by map them over an either object. Because the either object is a write object, the negate function will have no effect, and the add one function will be applied to the number five. Like functor, there's a kind of functors called profunctors that also takes two type arguments. The difference is a functor is covariant in both of its type arguments whereas a profunctors is contravariant in the first type argument and covariant in the second type argument. The diagram here is very similar to the diagram we saw earlier for functors, the only difference is that the arrow between type a and type b is flipped. A perhaps very helpful way for visualizing how a profunctors works is through a diagram like the one at the bottom. We can visualize a profounder p of a comma c as a box that takes an a and outputs a c. When we call dimap on the profounder with fng, we get back a new profounder that takes a b and outputs a d. There are many different profunctors. One of them is called forget it always returns some value of type r, no matter what type of input value you give it. Star and costar are two other kinds of profunctors. The symbol f here represents a functions. So starf takes an a and returns an f of c. When you die map on it, you can turn it into a new star f that takes a b and returns an f of D. Here's a cold example of the star profunctors. The star profunctors takes an integer and returns an option of int. If we give it integer three, it will give us sum of one because ten mod by three is one. If we pass zero to the profunctors, it will give us nothing because ten mod by zero is not a valid operation. We can take the star profunctors and die map on it with the stir to int function on the way in and with the is even function on the way out. What we end up with is a new profunctors that takes a string and returns an option of bool. As you can see, the code example we have starts to look like some sort of data pipelines that can be chained and compose in flexible ways. And that's indeed a good application of profunctors. Congratulations for making it to this point. We've covered a lot in a fairly short amount of time. In summary, we've introduced the concept of categories. Then we covered variant contravariant closed applicative functions as well as functors and profunctors. Hope you all enjoyed the talk. Thank you.
...

Chaur Wu

Freelance



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways