Conf42 Python 2022 - Online

How Decorators Function

Video size:

Abstract

Have you ever seen those “@” tags on top of Python functions and classes? Those are called decorators. Decorators, in their simplest form, wrap functions around functions. That might sound confusing at first, but it’s actually pretty useful. A simple decorator could measure execution times, add startup steps, or automatically repeat calls. Decorators are one of Python’s niftiest language features, and they help programmers write DRY (Don’t Repeat Yourself) code.

In this talk, we’ll learn all about decorators:

  1. How they wrap functions
  2. How to write our own decorators
  3. How to do cool tricks with arguments, classes, and nesting
  4. How to use popular decorators
  5. How to decide when decorators are (and aren’t) the right solution

We’ll walk through plenty of example code together. We’ll also touch lightly on Functional Programming (FP) and Aspect-Oriented Programming (AOP) concepts to build a firm understanding about how decorators work. After this talk, you should be able to use decorators effectively in your own Python projects!

Summary

  • Decorators are one of Python's niftiest language features. They wrap additional code around existing definitions. When used right, they can clean up your code better than oxiclean.
  • Decorators can be used to scrub and validate function arguments. They can also be applied directly to classes. Decorators should have their own separate unit tests. Tests will also be simpler if they use how decorators function.
  • When should you use decorators in your Python code? Use decorators for aspects an aspect is a special crosscutting concern. Good use cases for decorators include logging, profiling, input validation, retries.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hello everyone. Pandy Knight here, automation panda and developer advocate at applitools. I'm also a huge Pythons fan, just like y'all. Have you ever seen those at tags on top of python functions? Maybe you've seen them on top of methods and classes, too. Those are decorators, one of Python's niftiest language features. Decorators are essentially wrappers. They wrap additional code around existing definitions. When used right. They can clean up your code better than oxiclean. Let's learn how to use them. Here's a regular old hello world function. When we run it, it prints hello world. Nothing fancy. Now let's take that function and bam. Add a decorator using this at sign. We just added a decorator named Tracer to hello world. So what is this decorator? Tracer is just another function, but it's a special function because it takes in another function as an argument. Since tracer decorates hello world, the hello world function is passed into tracer as an argument. Wow. So what's inside tracer? This decorator has an inner function named wrapper. Can you even do that with Python? Yes, you can. The wrapper function prints entering, calls the function originally passed into the decorator, and then prints exiting. When tracer decorates hello world, that means hello world will be wrapped by entering and exiting print statements. Finally, the decorator returns the new wrapper function. Anytime the decorated function is called, it will effectively be replaced by this new wrapper function. So when we call hello world, the trace statements are now printed. Wow, that's amazing. That's how decorators work. Decorators wrap functions around functions. Think about them like candy bars. The decorators is like the foil wrapper. And how how decorators function like the chocolate inside. But how is those even possible? That decorator code looks confusing. Decorators are possible because in pythons, functions are objects. In fancy language, we say functions arent first order values. Since functions are just objects, we can pass them into other functions as arguments define new functions inside existing functions, and return a function from a function. Those is all part of a paradigm called functional programming. Python supports functional programming because functions can be treated like objects. That's awesome. So using functions as objects, decorators can change how functions are called. Decorators create an how decorators function around an inner decorated function. Remember, the outer function is like the foil wrapper and the inner function is like the chocolate. Creating an outer function lets you add new code around the inner function. Some people call this advice. You can add advice before or after the inner function. You could even skip the inner function. The best part is, decorators can be applied to any function. They make sharing code easy so you don't repeat yourself. Decorators are reminiscent of a paradigm called aspect oriented programming, in which code can be cleverly inserted before and after points of execution. Neat. So remember how decorators functions around functions like candy bars? Hold on, now we have a problem in that pythons code. If the wrapper function effectively replaces hello world, then what identity does hello world report? Its name is wrapper and its help is also wrapper. That's not right. Never fear, there's an easy solution. The functions module provides a decorator named wraps. Put functions wraps on the wrapper function and pass in the inner function object, and decorated functions once again show the right identity. That's awesome. But wait, there's another problem. How do decorators work with inputs and outputs? What if we decorate a function with parameters and a return value? If we try to use the current tracer, we get an error. Arguments broke it. Thankfully, we can fix it. First, add star args and startup kwargs to the wrapper function's parameters and then pass them through to the inner function. This will make sure all arguments go through the decorator into the decorated function. Then capture the inner function's return value and return it from the wrapper function. This makes sure return values also pass through. If the inner function has no return value. Dont worry, those decorator will pass through a none value. When we call the function with the updated tracer, we see tracing is now successful again. When we check the return value, it's exactly what we expected. It works. Wow, that's awesome. But wait, there's more. You can write a decorator to call a function twice. Start with the decorator template and call the inner function twice. Return the final return value for continuity. Bam. It works. Oh wait, there's more. You can write a timer decorators start with the template, call the inner function, and surround it with timestamps using the time module. Bam. Now you can time any functions. But wait, there's more. You can also add more than one decorator to a function. This is called nesting order matters. Decorators are executed in order of closeness to the inner function. So in this case, call twice is applied first, and then timer is applied. If these decorators are reversed, then each inner function call is timed. Cool. But wait, there's more. You can scrub and validate function arguments. Check out these two simple math functions. Create a decorator to scrub and validate inputs as integers. Add the wrapper function and make sure it has positional args. Then cast all args as ints before passing them into the inner function. Now, when calling those math functions calls, numbers are integers. Using non numeric inputs also raises a value error. But wait, there's more. You can create decorators with parameters. Here's a decorator that will repeat a function five times. The repeat function is a little different. Instead of taking in the inner function object, it takes in the parameter, which is the number of times to repeat the inner function. Inside there's a repeat decorator function that has a parameter for the inner function. The repeat function returns the repeat decorator function inside. Repeat decorator is the wrapper function. It uses functions, wraps, and passes through all arguments. Repeat decorator returns wrapper finally, wrapper contains the logic for calling the inner function multiple times according to the repeat decorator's parameter value. Now, hello, could runs five times. Nifty. But wait, there's more. Decorators can be used to save states. Here's a decorator that will count the number of times a functions is called. Could calls has the standard decorator structure outside the wrapper. A could attribute is initialized to zero. This attribute is added to the wrapper function object inside the wrapper. The count is incremented before calling the inner function. The count value will persist across multiple calls. Initially, the hello world count value is zero. After two calls, the count value goes up. Awesome. Oh wait, there's more. Decorators can be used in classes. Here those timer decorator is applied to this hello method. As long as parameters and return values are set up correctly, decorators can be applied equally to functions and methods. Decorators can also be applied directly to classes. When a decorator is applied to a class, it wraps the constructor. Note that it does not wrap each method in the class. Since decorators can wrap classes and methods in addition to functions, it would technically be more correct to say that decorators wrap callables around callables. So all that's great. But can decorators be tested? Good code must arguably be testable code well, today's your lucky day, because yes, you can test decorators. Testing decorators can be a challenge. We should always try to test the code we write, but decorators can be tricky. Here's some advice. First, separate tests for decorator functions from how decorators function. How how how decorators function intended outcomes try to focus on the wrapper instead of the inner function. Remember, decorators can be applied to any callable, so cover the parts that make the decorators unique. Decorate Ted functions should have their own separate unit tests. Second, apply decorators to fake functions used only for testing. These functions can be simple or mocked. That way, unit tests won't have dependencies on existing functions that could change. Tests will also be simpler if they use how decorators function. Third, make sure decorators have test coverage for every possible way it could be used. Cover decorator parameters, decorated function arguments, and return values. Make sure the name and help are correct. Check any side effects like saved state. Try it on methods and classes as well as functions with decorators. Most failures happen due to overlooked edge cases. Let's look at a few short decorator tests. We'll use the count calls decorator from earlier there are how how decorators function use for testing. The first one is a no operation function that does nothing. It has no parameters or returns. The second one is a function that takes in one argument and returns it. Both are very simple, but they represent two equivalent classes. How how how how decorators function will verify outcomes using the decorator for count calls. That means tests will focus on the count attribute added to the decorated function. The first test case verifies that the initial count value for any function is zero. The second test calls a function three times and verifies that the count is three. The third test exercises those same function to make sure arguments and return values work correctly. It calls the same function, asserts the return value, and asserts the count value. This collection of tests is by no means complete. It simplest shows how to start writing tests for decorators. It also shows that you don't need to overthink unit tests for decorators. Simple is better than complex. Up to this point, we've covered how to write your own decorators. However, Python has several decorators available in the language and in various packages that you can use absolutely free. Decorators like class method, static method, and projects can apply to methods in a class. Frameworks like flask and pytest have even more decorators. Let's take a closer look. Let's start by comparing the class method and static method decorators. We'll revisit the greeter class we saw before the class method decorator will return any method into a class method instead of an instance method. That means this hello method here can be called directly from the class instead of from an object of the class. The decorators will pass a reference to the class into the method, so the method will have some context of the class. Here, the reference is named CLS, and the method uses it to get the class's name. The method can be called using greeter hello. The static method decorator works almost the same as the class method decorator, except that it does not pass a reference to the class into the method. Notice how the method parameters are empty. No class and no self methods are still called from the class, like here with greeter goodbye. You could say that static method is just a simpler version of class method. Next, let's take a look at the property decorator to show how to use it. We'll create a class called accumulator to keep count of a tally. Accumulator's init method initializes a count attribute to zero. An add method adds an amount to the could. So far, nothing unusual. Now let's add a property. The count method has the property decorator on it. This means that counts will be callable as an attribute instead of a method, meaning that it won't need parentheses. It is effectively a getter calls to count in the init and add methods will actually call this projects. Instead of a raw variable inside those count property, the method returns an attribute named underscore count. The underscore means that this variable should be private. However, this class hasn't set that variable yet. That variable is set in the setter method. Setters are optional for projects. Here, the setter validates that the value to set is not negative. If the value is good, then it steps underscore count. However, if the value is negative, then it raises a value error. Underscore could is handled internally, while could is handled publicly as the property. The getter and setter controls added by the property decorator let you control how the property is handled. In this class, the setter protects the property against bad negative values, so let's use this class. When an accumulator object is constructed, its initial count is zero. After adding an amount to the object, the count goes up. Its count can be directly set to non negative values. Attempting to set the count directly to a negative value raises that exception. As expected, protection like that is great. Python packages also frequently contain decorators. For example, Flask is a very popular web micro framework that enables you to write web APIs with very little Python code. Here's an example. Hello World Flask app. Taken directly from the flask Docs online, it imports the flask module, creates the app, and defines a single endpoint at the root path that returns the string. Hello World Flask's app root decorator can turn any function into a web API endpoint. That's awesome. Another popular package with decorators is Pytest, Python's most popular test framework. One of Pytest's best features is the ability to parameterize test functions to run from multiple input combinations. Test parameters empower data driven testing for wider test coverage. To show how it works, we'll use a simple test for basic arithmetic test addition. It asserts that a plus b equals c. The values for a, b, and c must come from a list of tuples. For example, one plus two equals three, and so forth. The Pytest mark parameterized decorator connects the list of test values to the test function. It runs the test once for each tuple in the list, and it injects the tuple values into those test case as function arguments. The test case will run four times. Test parameters are a great way to rerun test logic without repeating test code, so act now before it's too late. When should you use decorators in your Python code? Use decorators for aspects an aspect is a special crosscutting concern. There are things that happen in many parts of the code, and they frequently require repetitive calls. Think about something like logging. If you want to add logging statements to different parts of the code, then you need to write multiple logging calls in all those places. Logging itself is one concern, but it crosscuts the whole code base. One solution for logging could be to use decorators, much like we saw earlier with the tracer decorator. Good use cases for decorators include logging, profiling, input validation, retries, and registries. These are things that typically require lots of extra calls inserted in duplicative ways. Ask yourself this, could the code wrap something else? If yes, then you have a good candidate for a decorator. However, decorators aren't good for all circumstances. You should avoid decorators for main behaviors because those should probably be put directly in the body of the decorated function. Avoid logic that's complicated or has heavy conditionals too, because simple is better than complex. You should also try to avoid completely sidestepping the decorated function that would confuse people. Ask yourself, this is the code you want to write, the wrapper or the candy bar itself? Wrappers make good decorators, but candy bars do not. I hope you found this infomercial about decorators useful. If you want to learn more, check out this real Python tutorial named Primer on Python decorators. It covers everything I showed here, plus more. Thank you very much much for listening again. My name is Pandy Knight. I'm the automation panda and a developer advocate at applitools. If you like my talk, then please read my blog and follow me on Twitter.
...

Andrew Knight

Developer Advocate @ Applitools

Andrew Knight's LinkedIn account Andrew Knight's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways