What is parallelism in programming?

Parallelism in programming is the execution of multiple tasks simultaneously or seemingly simultaneously, which helps improve the performance and responsiveness of applications.

There are several types of parallelism:

• Multiprocessing: Uses multiple independent processes, each with its own address space. This bypasses the GIL (Global Interpreter Lock) limitation in CPython and is suitable for CPU-intensive tasks.

  import multiprocessing
  import time
  
  def worker(num):
      """Some work"""
      time.sleep(1)
      print(f'Worker {num} finished')
  
  if __name__ == '__main__':
      processes = []
      for i in range(5):
          p = multiprocessing.Process(target=worker, args=(i,))
          processes.append(p)
          p.start()
  
      for p in processes:
          p.join()
      print("All workers finished")
  

• Multithreading: Uses multiple threads within a single process, sharing the same address space. Due to GIL in CPython, threads do not execute in parallel on different cores for CPU-bound tasks, but are effective for I/O-bound tasks.

  import threading
  import time
  
  def worker(num):
      """Some work"""
      time.sleep(1) # Simulate I/O operation
      print(f'Thread {num} finished')
  
  threads = []
  for i in range(5):
      t = threading.Thread(target=worker, args=(i,))
      threads.append(t)
      t.start()
  
  for t in threads:
      t.join()
  print("All threads finished")
  

• Asynchronous programming (Asyncio): Uses a single thread and an event loop to manage task execution. Suitable for high-load I/O-bound applications and web servers. Does not block execution while waiting for I/O.

  import asyncio
  import time
  
  async def worker(num):
      """Some asynchronous work"""
      await asyncio.sleep(1) # Asynchronous I/O wait
      print(f'Async worker {num} finished')
  
  async def main():
      tasks = []
      for i in range(5):
          task = asyncio.create_task(worker(i))
          tasks.append(task)
      await asyncio.gather(*tasks)
      print("All async workers finished")
  
  if __name__ == '__main__':
      asyncio.run(main())
  

The choice of approach depends on the nature of the task:

Approach	CPU-bound tasks	I/O-bound tasks	GIL	Memory usage
Multiprocessing	Excellent	Good	Bypasses	Large (own process)
Multithreading	Poor (due to GIL)	Excellent	Blocks	Small (shared process)
Asynchronous	Poor	Excellent	Does not block	Small (single thread)

Parallelism requires careful management of synchronization and resources to avoid race conditions and deadlocks.