Inter-Process Communication with multiprocessing.Queue

1Concept

Because processes do not share memory, communication requires Inter-Process Communication (IPC). `multiprocessing.Queue` serializes objects using `pickle` through an underlying OS pipe, providing thread-safe and process-safe message passing.

2Architecture Diagram

[ Worker Process 1 ] ---> Put (Pickles object into OS Pipe) ---> [ Main Process reads from Queue ]

3Code Example

import multiprocessing

def worker_task(queue: multiprocessing.Queue, task_id: int):
    queue.put(f"Result from worker {task_id}")

queue = multiprocessing.Queue()
worker_task(queue, 1)
worker_task(queue, 2)

print(f"Received from IPC Queue: {queue.get()}")
print(f"Received from IPC Queue: {queue.get()}")

4Expected Output

Received from IPC Queue: Result from worker 1
Received from IPC Queue: Result from worker 2

5Key Takeaways

• ✓Objects put into multiprocessing.Queue MUST be picklable.
• ✓Use `multiprocessing.JoinableQueue` to coordinate task completion with `.task_done()`.
• ✓Avoid placing huge data arrays in queues; use shared memory (`multiprocessing.shared_memory`) instead.