Sobes.tech
Junior — Middle

Which Python modules provide tools for implementing parallel task execution?

sobes.tech AI

Answer from AI

In Python, several core modules are used to implement parallel task execution:

  • threading — allows creating and managing threads. Suitable for I/O-bound tasks where parallelism is important, but does not increase CPU-bound task performance due to GIL.
  • multiprocessing — creates separate processes, bypassing GIL limitations, enabling efficient parallelization of CPU-bound tasks.
  • concurrent.futures — a high-level interface for working with threads (ThreadPoolExecutor) and processes (ProcessPoolExecutor), simplifying thread or process pool management.
  • asyncio — for asynchronous programming, allows executing tasks concurrently without creating threads or processes, suitable for I/O.

Example of using concurrent.futures with threads:

import concurrent.futures
import time

def task(n):
    time.sleep(n)
    return f"Task {n} completed"

with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
    futures = [executor.submit(task, i) for i in range(3)]
    for future in concurrent.futures.as_completed(futures):
        print(future.result())