Advanced Python: Decorators Simplified

Table of Contents

Introduction

Decorators in Python are a metaprogramming technique, in the sense that we are manipulating the program code itself using our code. A decorator is used in order to extend the functionality of an object, without explicitly modifying it. Before we dive into the subject, we have to understand the concept of first class functions.

First Class Functions

Functions in Python are treated as first-class citizens. This means we can perform basic operations on functions like assigning them to variables, returning them, passing them as arguments, and other basic operations that we can do with other objects in the language.

Passing functions as arguments:

We pass function bar (without running it) to function foo, so it can execute bar:

def foo(arg_func):
    print('I will run the argument function!')
    arg_func()
    print('Run done!')


def bar():
    print('Function bar has been run')


foo(bar)

I will run the argument function!
Function bar has been run
Run done!

Assigning functions into variables

We can store functions in variables and use them later:

def foo():
    print('Foo has run')


var = foo
var()

Foo has run

Returning functions from functions

We can also return functions from functions:

def foo():
    def bar():
        print('Bar')
    return bar


return_val = foo()
return_val()

Bar

Callables in Python

A callable in Python is any object that can be called. These can be functions, but can also be any classes that implement the __call__ method. This method is called when we invoke the () operator on the object.

class CallableClass:
    def __call__(self):
        print('We called the class')


callable_obj = CallableClass()
callable_obj()

We called the class

Notice we called the instance of the class we created, as if it was a function.

Python Decorators

a Python decorator is a callable, that gets a callable, and returns a callable as its return value.

We define decorators as callables because they can either be functions or classes. Let’s look at both cases.

Function Decorators

We define a decorator function as a function that receives a callable as input. The decorator function defines a new wrapper function in order to modify the behavior of the input function, and returns the new wrapper function as output.

def decorator(func):

    def wrapper():
        # Optional: Do something before
        print('Behavior extended to printing before running')

        # Execute function
        func()

        # Optional: Do something after
        print('Behavior extended to printing after running')
    return wrapper


def foo():
    print('Foo has run')


foo = decorator(foo)
foo()

Make sure you understand the above example. We define a simple function named foo, that performs a print.
In our decorator function, we define a wrapper function that is returned, and executes the input function, while also doing some operations before and/or after. Because of this, when we run foo = decorator(foo), foo’s behavior has been extended to also include the wrappers function behavior.

Running the code gives us the output:

Behavior extended to printing before running
Foo has run
Behavior extended to printing after running

Instead of executing foo = decorator(foo), we could have also defined foo this way:

@decorator
def foo():
    print('Foo has run')

And if we run foo:

Behavior extended to printing before running
Foo has run
Behavior extended to printing after running

This achieves the same result. This is the preferred way as it keeps our code clean and readable, even though under the hood it performs the same operation foo = decorator(foo). This shorthand writing is an example of a so called syntactic sugar.

Functions identity has changed

If you look at some of the built in metadata attributes of our function after the decoration (for example __name__) you’ll notice something has happened:

print(foo.__name__)

wrapper

foo’s name has changed to wrapper, because the function itself was changed to the wrapper function because of the return value. This may not be so bad, however if you are debugging or writing the function’s name into logs for example, this could confuse you.

@functools.wraps to the rescue

In order to prevent the loss of the decorated functions metadata, we can use the @functools.wraps decorator on the wrapper function:

def decorator(func):

    @functools.wraps(func)
    def wrapper():
        # Optional: Do something before
        print('Behavior extended to printing before running')

        func()

        # Optional: Do something after
        print('Behavior extended to printing after running')
    return wrapper

This decorator is responsible for preserving the decorated function’s metadata. If we check the metadata of our function now:

print(foo.__name__)

foo

Class Decorators

Just like functions, classes can also be decorators. This means they receive a function as input just like before, and will extend the functionality when being called. Classes can be excellent when you want to keep a state, which you can read and update the next time it is being called.

Let’s define a decorator class that logs how long a function has run, but also keeps as a state the longest execution time of all the times it was executed.

from time import time


class MaxTimer:
    def __init__(self, func):
        functools.update_wrapper(self, func)
        self.func = func
        self.longest_execution_time = 0.0

    def __call__(self):
        time_before_execution = time()
        ret_val = self.func()
        time_after_execution = time()

        self.longest_execution_time = max(self.longest_execution_time,
                                          time_after_execution - time_before_execution)
        print(f'{self.func.__name__} has taken {(time_after_execution - time_before_execution):.2f} seconds to execute')
        print(f'{self.func.__name__} longest execution time is {self.longest_execution_time:.2f} seconds')
        print()

        return ret_val


@MaxTimer
def foo():
    sum = 0
    for i in range(50000000):
        sum += 1

    return sum


for i in range(3):
    foo()

Remember that when we’re using @MaxTimer on foo(), it’s the same as running foo = MaxTimer(foo). This is why our class has to accept foo as an input argument in the constructor, and also implement the __call__ function so we can actually call the function foo after annotating with @MaxTimer.

In our constructor we are saving a reference to our input function so we can call it in __call__, and also keeping a data member that will keep the state: what is the longest time the function took to execute. In our call function we are just performing the benchmark and saving the max result.

By running the above code, we are calling foo 3 times. This is an example of the output:

foo has taken 3.43 seconds to execute
foo longest execution time is 3.43 seconds

foo has taken 3.35 seconds to execute
foo longest execution time is 3.43 seconds

foo has taken 3.36 seconds to execute
foo longest execution time is 3.43 seconds

Using multiple decorators

It is possible to use multiple decorators on the same function. Let’s look at an example and explain the order of execution:

def one(func):
    @functools.wraps(func)
    def wrapper():
        print('One: pre execution')
        func()
        print('One: post execution')

    return wrapper


def two(func):
    @functools.wraps(func)
    def wrapper():
        print('Two: pre execution')
        func()
        print('Two: post execution')

    return wrapper


@two
@one
def foo():
    print('Function executed')


foo()

Two: pre execution
One: pre execution
Function executed
One: post execution
Two: post execution

Remember that this is the same as running foo = two(one(foo)). We first extend foo’s behavior with @one, adding one’s pre and post behavior. The result’s behavior is being extended with with @two, adding two’s pre execution logic – before the execution of one, and post execution logic – after the execution of one.

Decorators of functions with parameters

In all the examples above our functions didn’t get any arguments. What if we want to decorate a function that gets arguments? We would have to make sure the wrapper function that we return can get the same arguments, but we want to be generic so the decorator can be used on any function regardless of the number of the parameters. For example, in the case of our MaxTimer example, we may want to use it on many different functions in our codebase even if they accept 1, 5 or 10 parameters.

This can be achieved using *args and **kwargs which is python’s generic way to annotate a dynamic number of arguments and keyword arguments. We define wrapper so it gets both of these arguments, and passes these on to the function to execute.

def params_example(func):

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print('Pre execution')
        func(*args, **kwargs)
        print('Post execution')

    return wrapper


@params_example
def foo(print_yes):
    if print_yes:
        print('Yes')
    else:
        print('No')

Now we can run foo with True as input:

foo(True)

Pre execution
Yes
Post execution

And also with False as input:

foo(False)

Pre execution
No
Post execution

Built-in decorators in Python

Reading this, you may have recalled decorators that you may have used in the past, without knowing they are such. Here are a few examples:

@abstractmethod – Declares an abstract method, which will throw an exception if not implemented by overriding.
@staticmethod – Declares a static method, which means the method will not get the self parameter as an argument. The method will not have access to a specific instance of the class.
@classmethod – This is the same as @staticmethod, however the function also gets the class object as a parameter