Leverage the abstractions and composability of functors. Do you want to use covariant or contravariant functors to lift regular Python functions and compose them without side effects? Interested in building highly extensible and composable data pipelines with pro-functors? This talk will show you how.

- In this talk, we will look at functions and how to use them in Python programs. This talk requires no prior knowledge of functions or category theory. The talk is for Python programmers and is designed for ease of understanding.
- To visualize the code, we can draw types as dots and functions as arrows. A diagram like the one we are seeing here represents a category in category theory. Let's move on to see how functions are related to categories.
- In python, we can combine categories and functions to create new functions. With the I O function, we are able to keep the kernel of our code free of side effects. This solution can turn a partial function into a total function while retaining composability.
- Next, let's turn our attention to a kind of functions called functors in Python. In python code, a functor class takes only one type argument, whereas a profunctors class takes two type arguments. A perhaps very helpful way for visualizing how aprofunctors works is through a diagram.
- In summary, we've introduced the concept of categories. Then we covered variant contravariant closed applicative functions as well as functors and profunctors. Hope you all enjoyed the talk.

Hi, thank you for coming to my talk. In this talk,
we will look at functions and how to use them in Python programs.
Don't worry if you haven't heard of functions before.
This talk requires no prior knowledge of functions or
category theory. The talk is for Python
programmers and is designed for ease of understanding.
There will be places where we will choose to simplify things at
the expense of skipping some math details.
The code examples are based on an open source Python package called
funklift. The links to the GitHub repositories
of funklift and its tutorials are shown on
the slide. Before we move on
to the meat of the talk, a quick introduction of myself.
My name is Char Wu. I'm a software developer,
grew up in Taiwan, and I'm based in the San Francisco Bay
area for the past 20 years.
Now, today's agenda is very simple. We will
start with a motivating example and then we will
introduce various functions with even more examples.
Let's start with a simple function, get price that takes
an item and returns the item's price.
To visualize the code, we can draw types as dots and
functions as arrows. As you can see,
item is a dot in the diagram and int is also a dot.
Get price is an arrow that goes from item to
int. Now we have another function.
It's pricey and we can draw it as an arrow
as well. This time the arrow goes from int
to bool. With those two functions,
what interesting things can we do with them?
Well, we can compose the two functions to form a
new function. Notice that the composition
of functions in code corresponds
nicely to the composition of arrows in our diagram.
A diagram like the one we are seeing here represents a
category in category theory.
Essentially, a category consists of dots and arrows,
plus some properties about how arrows compose.
We will call our category here the
category of types. In this category,
the dots are Python types and the arrows are Python
functions. Now let's
move on to see how functions are
related to categories.
Here we have a function is even that goes from into
pool and we have a list of numbers.
1234 we want to map is
even over the numbers. One way to
achieve that is by using this comprehension.
Another way is to use the CDS class of the
funklift package.
As you can see from the code here, the CDS class
has a method called fmap.
Fmap takes the function is even and maps
it over the list of numbers behind the scenes.
On the outside, we can think of fmap as
lifting is even to a function that goes
from c list of int to c list of bool.
If we put the is even function and its
lifted counterpart side by side, we can see
that C list is actually a mapping from a source
category to a target category.
Such a mapping is called a functor if
it satisfies certain properties. In our
case, C list is a functor. The source
category and the target category of C list are
both the category of types. A functor
is called an functor when its source and
target categories are the same.
Okay, now that we've introduced
the concepts of categories and functions,
let's see some functions in
action, and let's also see
what problems they solve.
It's common to have code
that performs some kind of input or output. For example,
we might have code that sends a request over a network,
reads a file, or writes a record to a database.
Such I O actions are side effects
that make our code harder to reason about because they
break referential transparency.
Here on the side, the function get number side
effect on the left has side effects because it reads user
input from a console.
To avoid such side effect, we can wrap
the I O action into an instance of the I O class.
With that, when we call the get number
function on the right, no actual reading
from the console will take place.
Rather, we simply get an I O object that represents
the I O action.
The code here shows what we can do with the I O object
returned by the get number function, even though
no actual I O has happened yet. When we call get
number, that does not stop us from mapping the is
even function over the nonexistent number.
Eventually, when we are finally ready to incur
the actual I o, we would do so by
coding the unsafe run method on
the I o object.
Although our example here is simple already, it shows that with
the I O function, we are able to keep the kernel of
our code free of side effects and
push the actual I o actions to the boundary of our
program.
Now let's look at another example here.
At the top we have a function that takes an integer and returns
ten modulo that integer. The function
is a partial functions because it's not defined when
the input integer is zero.
To fix that, we can write the function at the bottom.
The function at the bottom is now defined for all input values.
However, we now have a different issue, and that is
the new function is not very composable with other functions.
Here's an illustration of the lack of composability.
As you can see, the code on the right tries
to compose the two functions on the left,
because the ten mark by functions returns either
an integer or none. It is not very composable
and we have to use if else statements when we try to compose
it with other functions. Is there a
solution that can turn a partial
function into a total function while retaining composability?
Yes, that solution is the option functions.
As the example here shows, the ten
mark by function now returns an option of int.
The option class of the funklift package has
a subclass called nothing and another subclass
called sum. Nothing represents the
absence of a value. On the other hand,
sum represents the existence of a value.
By the magic of functions and their fmap methods,
we can compose ten map by with other functions
without any if else statements in the code.
Just like how we visualize c list as a functor,
we can visualize option as a functor.
The functor maps dots and arrows from
a source category to a target category.
So far we've been composing functions.
It turns out that we can compose functions too.
This diagram shows that we can compose the option
functor and the c list function.
The result of the composition is a new functions,
and we call that new functions c list after
option. Like any other functions,
the new functions is a mapping between two categories.
It maps dots to dots and arrows to arrows.
If we can compose functors the way we compose
arrows, does that mean functors are arrows
in some sort of category?
Notice that in this diagram, each rectangle
represents a category. If we collapse
the rectangles into dots, we will
get a category whose dots are smaller
categories and whose arrows are functions.
Here's what that category looks like in a diagram.
We call it the category of small categories.
Of course, we don't just look at theories,
we want to see some code, right? Absolutely.
The code example on this side shows how to compose functions
in python. Here, the variable nums
is a c list of option objects.
If we call fmap on nums, we will
not be able to map over the numbers inside the option objects.
In order to do that, we need to compose the two functions
first to form a new functions, and we do that by
using the compose class of the funklift package.
Once we have the composite functions, we can call
fmap on it, just like we do with any other functions.
This is really fantastic.
In this diagram, capital f denotes a functions.
The functions we've seen so far are called covariant functions.
Here, contravariant basically means
that the two vertical arrows in
the diagram point in the same direction. In other
words, when a functor maps a source
arrow to a target arrow, if the mapping does not change
the direction of the arrows, then the functor
is covariant.
On the other hand, if the two vertical arrows
in the diagram point in the opposite direction,
then the functions is contravariant.
For covariant functions we have the fmap method.
For contravariant functions we have the cmap method.
Similar to fmap, the cmap method
lifts a function from type a to type b to
a function that goes from f of b to f of
a. Here's an
example of a contravariant function called predicate.
A predicate is something that is either true or false.
In our example, we first have a predicate
that will be true if we give it an even
integer, and as the diagram shows,
we can use cmap to convert that predicate of int into
a predicate of str.
With the converted predicate we can pass it the
number six as a string, and it will tell us
if that's an even number or not.
The next type of functions we will look at is called
applicative functions. Before we
introduce applicative functor, we need to first get to know
a special kind of categories called closed category.
In the diagram on the side, there are two categories.
The category on the left has a dot
for type a and a dot for type b.
If the category also has a dot in
gray for the functions type a to b,
plus some other properties, then the
category is called a closed category.
An example of a function type is into bool,
as shown in the blue box.
Now let's turn our attention to the category on the right.
The category has a dot for the type
f of a and a dot for the type f of b.
Because it's also a closed category, it has a
dot for the function type f of a to
f of b in green. Between the
two categories we have a functions that maps dots to
dots and arrows to arrows. In particular,
the functor maps the gray dot a to b
on the left to the blue dot,
f of a to b on the right in blue.
Now, it may be the case that the blue dot
and the green dot are not related at all.
However, if they happen to be one and the same
dot, then we say that the functor preserves
the structure of the source category, and we
call the functor a strict closed functor.
Requiring the blue dot and the green dot to be the same dot is a
rather strict condition. A somewhat more
relaxed condition is to
not require the two dots to be the same.
Instead, we merely require that there's an
arrow between the two dots in such a case
we call the functions a lex closed functions.
Another name for another
name for next for next closed functions is
applicative functions,
and we call the arrow between
the blue dot and the green dot app.
That's a lot of theory. Let's see an example in
code.
Here we have a curried function for summing two integers.
Currying means if a function takes multiple arguments,
then the current function will take one argument at a time.
In this example, we start with sum
of 20. Then we f map the current
read function over it.
What we end up with is an option of end
to end, which is shown as the
blue dot in the diagram. Because our option
functions is an applicative function, we have the
app arrow that takes us from the blue dot to the
green dot. The green dot
is essentially a function that takes an option of
int and returns another option of int. So if
we give that function sum of 30, we will get back
sum of 50 as the result.
Next, let's turn our attention to a kind of functions
called functors in Python.
If we have a type a and a type c,
we can form the tuple type a comma c.
Similarly, if we have a type b and a type
d, we can form a tuple type
b comma d. And if
there's an arrow from a to b and another
arrow from c to d, we can take those two arrows
and form a tuple of arrows that goes from
a comma c to b comma d.
Now, if we have a functions that maps a comma c
to f of a comma c and b comma
d to f of b comma d,
and the functor also maps the tuple of arrows to
a lifted tuple of arrows, then we call
such functor a functor. In python code,
a functor class takes only one type argument,
whereas a functor class takes two type arguments.
Let's see an example of a functor. The functor
in the example is the either class from the funk devt
package. As its name suggests,
either represents one of two possibilities.
We represent those two possibilities with two classes
left and right. In the coexample,
we take two functions, add one and negate,
and we by map them
over an either object.
Because the either object is a write object,
the negate function will have no effect, and the
add one function will be applied to the number five.
Like functor, there's a kind of functors called profunctors
that also takes two type arguments. The difference
is a functor is covariant in
both of its type arguments whereas a profunctors
is contravariant in the first type argument and
covariant in the second type argument.
The diagram here is very similar to the diagram we
saw earlier for functors, the only difference is
that the arrow between type a and type
b is flipped.
A perhaps very helpful way for visualizing how
a profunctors works is through a diagram like
the one at the bottom. We can visualize
a profounder p of a
comma c as a box that takes an a and
outputs a c. When we call dimap
on the profounder with fng,
we get back a new profounder that takes
a b and outputs a d.
There are many different profunctors. One of them is called
forget it always returns some
value of type r, no matter what type of
input value you give it.
Star and costar are two other kinds of profunctors.
The symbol f here represents a functions.
So starf takes an a and
returns an f of c.
When you die map on it, you can turn it into
a new star f that takes a b and returns
an f of D.
Here's a cold example of the star profunctors.
The star profunctors takes an integer and returns
an option of int. If we give
it integer three, it will give us sum of one
because ten mod by three is one.
If we pass zero to the profunctors,
it will give us nothing because ten mod by zero is not
a valid operation.
We can take the star profunctors and die map
on it with the stir to int
function on the way in and with the is
even function on the way out.
What we end up with is a new profunctors that
takes a string and returns an option of bool.
As you can see, the code example we have
starts to look like some sort of data pipelines that
can be chained and compose in flexible ways.
And that's indeed a good application of profunctors.
Congratulations for making it to this point.
We've covered a lot in a fairly short amount of time.
In summary, we've introduced the concept
of categories. Then we covered variant
contravariant closed applicative functions as
well as functors and profunctors.
Hope you all enjoyed the talk. Thank you.

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways