Python Asyncio – Multithreading and Asynchronous Best Practices with example

Today in this example we will learn how to perform Python threading with a few examples and we will also learn the Best Practices.

We will see how to optimize code execution by performing Python asynchronous code execution.

We will cover the below aspects in today’s article,

Asynchronous programming is a programming allows tasks to be executed concurrently and independently of each other, without blocking the execution of the main program. In asynchronous programming, tasks are executed asynchronously, meaning they can start, run, and complete independently of the main program’s execution flow.

The key concept in asynchronous programming is the use of non-blocking operations, where a program can initiate a task and continue executing other tasks without waiting for the completion of the initiated task. This enables better utilization of resources and improved responsiveness in applications, especially when dealing with I/O-bound operations such as network requests, file I/O, and database queries.

Getting Started – Multithreading with synchronous(Blocking) and asynchronous(non-bloking) execution

Before we deep dive into the Python multithreading example, please note that there are multiple ways one can obtain Multithreaded execution in Python.

Main techniques inlcues like using Python Threading vs ThreadPoolExecutor Vs Asynco Python.

Let’s take a simple use case using a Python application where I have one operation that needs to be Multithreading with parameters in Python.

I have a requirement to invoke multiple service calls and combine their decisions to display to the user as if like a single operation.

However, the below concept discussed is generic enough and can be applied to any Python business logic as well.

I have the below IDs to be processed asynchronously.

Each of above the IDs needs to be processed by calling an HTTP API method.

The below API processes the above IDs and provides the result,

Let’s now see how to launch each request sequentially and parallelly and achieve the result.

Old – Executing Synchronous method in Python

I have a simple example where a service request takes a few seconds to execute.

# Define an synchronous function to be executed
def perform_task(parameter):
    # Start time
    start_time = time.time()
    print(f"Task executing with parameter: {parameter}")

     # Simulate some asynchronous work by calling Sleep 
    time.sleep(5) 

     # End time
    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Operation took {elapsed_time} seconds")
    print("Task execution complete")

Since we are executing the method synchronously, each operation executes in order and one after the other.

Below is the main function

def main():
    # Define the parameters
    parameters = ["param1", "param2", "param3"]
    # Start time
    start_time = time.time()
    # Create and gather tasks for each parameter
     # Perform the task for each parameter
    for param in parameters:
        perform_task(param)
      # End time
    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Task execution complete . Total Time taken : {elapsed_time} seconds")

if __name__ == "__main__":
      main()

So short, each call is a blocking call and takes its own time before proceeding to the other.

Each synchronous operation executes, so the total time required to complete the operation is equal to the sum of all requests processed operations.

The total time required to complete all operations is ~ 15 second

Below is the code base for Synchronous operation,

Asynchronous Task execution in Python using Asyncio

Please add below import statement below in your Python file

import asyncio
import time

Let’s declare the above operation using the asyncio Python example

A main() method as below,

async def main():
    # Define the parameters
    parameters = ["param1", "param2", "param3"]
    # Start time
    start_time = time.time()
    # Create and gather tasks for each parameter
    tasks = [perform_task(param) for param in parameters]
    await asyncio.gather(*tasks)
      # End time
    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Task execution complete . Total Time taken : {elapsed_time} seconds")

if __name__ == "__main__":
    asyncio.run(main())

In this example:

• We define an asynchronous method perform_task that represents the asynchronous task to be executed by each coroutine.
• We use asyncio.gather() to run all coroutines concurrently.
• We used asyncio.run() to run the main coroutine.
• We measure the total time taken for the execution by recording the start time before running the main coroutine and the end time after it completes. Then, we calculate and print the elapsed time.

Finally, the result is displayed below,

Each operation executes parallel and the maximum time taken by any operation would be the actual time taken by the whole operation.

So all the method execution gets completed or content around the same time as one of the operations.

However, timing may vary depending on CPU, CORE, and RAM memory GB size as needed.

Increasing the load

Now even if I increase the request load by putting more request execution you will see the total time required would remain near the same time of execution and allowing us to get better throughout overall.

However, as highlighted above this also depends on the CPU load at the time of code execution and also depends on the CPU used by the machine, and its hardware configuration

Note- Use must use async and await throughout your codebase consistently get the full benefit of asynchronous execution.

Do you have any comments or ideas or any better suggestions to share?

Please sound off your comments below.

Happy Coding !!

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published(2024) best practices and guidelines for software design and development.