Using callbacks in any programming language can go sideways very quickly and create chains of functions calling functions that call other functions that get to the point where you can't follow the logic anymore. In this post I will be focusing on what callback's are, how they can be useful, and general guidelines that I have found to allow you to use this methodology to solve problems without shooting yourself in the foot.

What is a callback

A callback is simply when you pass an unexecuted function or method as the argument to another function or method which in turn is called at some point during the second functions execution. To put a different way, it is the act of passing unexecuted code as an argument to some other code, which in turn will run the unexecuted code that was passed in at some point.

This is better illustrated in code so lets take a look below, I define my callback function.

def my_callback():
    print('my_callback')

And then I define a second function which will be the function that makes the call back to the callback function.

def second_function(callback):
    print('second_function')
    callback()

This function executes its own code first, which in this case is just a print statement and then executes the callback function that was passed into it.

And lastly to tie it all together, we call the second_function and pass in the my_callback function in one line.

if __name__ == '__main__':
    second_function(my_callback)

Notice that I do not have parenthesis after my_callback which means that we have not executed the function, we are just passing in an instance of it.

And when we execute this script we get the following output.

$ python callbacks.py
second_function
my_callback
$

And as you can see the second function properly made the call to the callback function executing its code.

Its important to keep in mind if you are new to this concept at this time this is a very simple example and while not very practical, I will explain later on how this concept can be very powerful.

Where are callbacks used typically

In the example above it is executed in a single threaded application and using a callback in that context doesn't make a whole lot of sense when you could just call each function one after the other directly. I would argue in a single threaded application like the example above this would be a very poor choice because there is added complexity in the logic for no good reason.

Where things are different in my opinion are in multi-threaded applications specifically GUI applications written in Tkinter or any event driven programming framework. Because you need to implement long running functions in threads in order to prevent locking up the GUI, callbacks can be very useful to start up a secondary thread from your main thread, pass in a call back and just let your main thread hang out and wait and the secondary thread makes the call to the callback function in the main thread and your program can move on from there.

The key point here is to keep your callback chains as short as possible. For example you should make every effort to not pass a callback to a function, which then in turn passes a different callback to a different function and so on as to create these chains of callbacks. Future you will not like present you if you do that.

Using Callbacks Effectively

In this example I will be using multiple threads so if you are not familiar with multi-threading in Python I advise you to familiarize yourself with threading in Python before continuing.

First let's import some libraries we will need for this example

from threading import Thread
from time import sleep
from random import randint

Then let's define our first thread which we will consider our "working thread" which will be the main orchestrator of what happens in our script.

class WorkingThread(Thread):

    def __init__(self):
        super().__init__()
        self.ancillary_complete = False

    def on_completion(self):
        print('AncillaryTask complete')
        self.ancillary_complete = True

    def run(self) -> None:
        x = AncillaryTask(self.on_completion)
        x.start()
        while not self.ancillary_complete:
            sleep(1)
        print('WorkingThread complete')

Stepping through this object, it inherits from Thread so it will execute in its own thread.

We override the __init__ method, accepting no parameters, and define an instance variable self.ancillary_complete and set it to False.

Next, we define the on_completion method which is our callback. This method is designed to only be called as a callback from a different method or function, when called it makes a print statement and then sets the instance variable self.ancillary_complete to True. The key here is to understand that even though an outside thread will call this method, it can update the state of the callback object, this means that the method we pass this callback to needs to know nothing of the variables it is setting, just that it needs to execute this callback. 

Lastly, we define the run method which is the logic of the thread when started. This run method starts an AncillaryTask object, which is another thread itself. And then drops in a loop waiting for the self.ancillary_complete instance variable to be set to true, and then printing that the thread is complete.

Now we need to define the AncillaryTask object.

class AncillaryTask(Thread):

    def __init__(self, on_complete_callback):
        super().__init__()
        self.on_complete_callback = on_complete_callback

    def run(self) -> None:
        sleep(randint(1, 10))
        self.on_complete_callback()

Stepping through this object, it too inherits from the Thread object making it run in its own thread.

We define the __init__ method and accept one parameter which is the callback to execute when this thread is complete. __init__ is also called in the parent object, passing no parameters to it, and then we store the callback in an instance variable so other methods of the class can access it.

The the run method is defined, and for the sake of example it just waits a random number of seconds between 1 and 9 seconds, and then executes the callback method that was passed into the object.

Lastly to tie these 2 objects together we write a simple script

if __name__ == '__main__':
    m = WorkingThread()
    m.start()
    m.join()
    print('script complete')

This simply instantiates the WorkingThread object, starts it, waits for it to finish, and the prints "script complete"

So finally with all that in place if we execute the script we will get the following output

$ python callback_example.py
AncillaryTask complete
WorkingThread complete
script complete
$

And you can see that the WorkingThread started its execution, then in its execution it instantiated and started the AncillaryTask object, passing in its own method as a callback.

Once the AncillaryTask method completed, it printed that it has completed and then executed the callback that was passed in by the WorkingThread object.

That callback as described above set the instance variable to True inside of the WorkingThread object allowing it to break out of its own loop and complete.

And lastly the main thread of the application was able to print out "script complete" and exit.

Key Concept

Because I believe it is such an important concept, I want to linger a bit on why using a callback in this scenario is very powerful. We could have coded our ancillary task to accept the calling object itself and update the state directly or done some other method of having the ancillary task manipulate the calling object directly. The power here is that now our ancillary task is not tied to this implementation, we can use this object in other parts of our codebase and maintain its functionality 100% and if on completion it needs to do something completely different, we change zero code in our ancillary task, just pass in a different callback. This creates a sort of interface for our object and allows for greater code reuse.

Closing

While this logic is less straight forward to follow than a single threaded application, I hope that I was able to shed some light on how callbacks can be useful and inspire you on how you could use them in event driven programming. Things I can think of is passing callbacks to buttons in Tkinter that get executed when someone presses a button, or you have a bunch of things that need to be processed and they need to show in a listbox, you can have the calling method execute the callback method after every loop in processing and pass in data to it which allows your listbox to be updated. And yes you can pass in objects to the callbacks, they just need to be built to accept parameters.

The options are limited only by your imagination, the one thing I want to stress highly is if at all possible keep your callbacks 1 layer deep, have a single thread be the main orchestrator and the other threads call back to its methods versus making a chain of callbacks.