Junior — Middle
Describe your experience working with multi-processor data processing systems and process interaction.
sobes.tech AI
Answer from AI
In Python, I used the multiprocessing module for parallel data processing, which allows running multiple processes simultaneously, effectively utilizing multiple CPU cores. For inter-process communication, I used queues (Queue) and pipes (Pipe) to transfer data and synchronize work.
For example, to process a large data array, I created a process pool (Pool), distributed tasks across processes, and collected results:
from multiprocessing import Pool
def process_data(item):
return item * item
if __name__ == '__main__':
data = [1, 2, 3, 4, 5]
with Pool(processes=3) as pool:
results = pool.map(process_data, data)
print(results) # [1, 4, 9, 16, 25]
I also used locks (Lock) to prevent conflicts when accessing shared resources. This approach allows efficient parallelization of tasks and improves application performance.