Junior — Middle
What methods can be used to achieve concurrent execution of threads in Python, considering the GIL limitation?
sobes.tech AI
Answer from AI
In Python, the Global Interpreter Lock (GIL) restricts the concurrent execution of bytecode in multiple threads, which makes parallelism challenging in CPU-bound tasks. To achieve concurrent thread execution, you can use:
- The
multiprocessingmodule — creates separate processes, each with its own interpreter and memory, which bypasses the GIL. - Use threads (
threading) for I/O-bound tasks, where threads wait for input/output and the GIL is not a bottleneck. - Use libraries that perform computations in C/C++ without holding the GIL (e.g., NumPy), allowing parallel execution of calculations.
Example with multiprocessing:
from multiprocessing import Pool
def f(x):
return x * x
if __name__ == '__main__':
with Pool(4) as p:
print(p.map(f, [1, 2, 3, 4]))
Thus, for CPU-bound tasks, it is preferable to use processes, and for I/O-bound tasks, threads.