The Python single-node performance ecosystem¶

Objectives

Distinguish concurrency from parallel execution.
Explain how the GIL affects CPU-bound Python code.
Choose among threads, processes, native code, and Dask for a workload.

Instructor note

20 min teaching/type-along
15 min exercises and discussion

Modern nodes gain throughput mainly from multiple cores and vector units. Python does not automatically use those resources, and dividing work creates scheduling, communication, and memory costs. Start by identifying the unit of work and the resource that limits it.

Four useful models¶

Model	Memory	Good first fit	Main cost or risk
Python threads	Shared	Waiting for files, networks, or other I/O	The GIL limits parallel Python bytecode
Native-library threads	Shared	NumPy/BLAS or compiled kernels that release the GIL	Oversubscription and race conditions
Python processes	Separate	Independent CPU-bound Python tasks	Serialization, copying, and process startup
Dask local scheduling	Threads or processes	A graph of tasks or chunked array/table work	Scheduler overhead and poor chunking

Concurrency means multiple tasks can make progress. Parallelism means work is executing at the same instant on multiple resources. Threads can provide useful concurrency even when they cannot execute Python bytecode in parallel.

The Global Interpreter Lock¶

In the standard CPython runtime, a thread must hold the Global Interpreter Lock (GIL) while executing Python bytecode. Two threads in one process therefore do not normally accelerate a CPU-bound loop written in Python.

The important boundary is not “Python versus threads,” but who executes the hot work:

a Python loop typically holds the GIL;
I/O releases the thread while it waits;
many NumPy, compression, and other native kernels release the GIL;
a separate process has its own interpreter and GIL.

This is why a threaded Dask scheduler can accelerate a native array kernel but fail to accelerate the same loop expressed as Python objects.

Runtime costs¶

The serial-work bound introduced in episode 1 is only the ideal limit. Real execution also adds task creation, scheduling, data transfer, allocation, and synchronization. A thousand microsecond tasks are often worse than a few tasks that each do substantial work. More workers can also make performance worse when each worker starts its own multithreaded BLAS kernel.

Control oversubscription

For predictable measurements, record worker counts and native thread settings such as OMP_NUM_THREADS, MKL_NUM_THREADS, or OPENBLAS_NUM_THREADS. A common starting point is one native thread per process.

Decision exercise¶

Exercise

Choose a first implementation for each workload and name one measurement that could change your decision.

Download 500 independent files.
Apply a NumPy universal function to four large array chunks.
Run a CPU-heavy pure-Python parser on 100 independent files.
Accelerate one deeply nested numerical loop called thousands of times.

Solution

Threads are a good first fit because the workload waits for I/O. Measure network/server saturation.
Threads or Dask’s threaded scheduler may work because NumPy commonly releases the GIL. Measure memory-bandwidth saturation and native-library threads.
Processes or Dask’s process/distributed scheduler avoid the single-process GIL. Measure serialization and task duration.
First improve the kernel with Cython (or an appropriate native implementation). Parallel scheduling cannot remove interpreter overhead inside each loop iteration.

Race-condition exercise¶

Exercise

Why is “threads share memory” both an advantage and a risk? Describe a safer shape for a parallel sum than having every worker update one shared total.

Solution

Sharing avoids serialization and copies, but unsynchronized read-modify-write operations can lose updates. Give each worker a private partial sum and combine the partials in a controlled reduction. This also reduces synchronization frequency.

Keypoints

Choose an execution model from the bottleneck and data-sharing needs.
The GIL limits simultaneous Python bytecode, not all native work called from Python.
Processes bypass the single-process GIL but add startup and data-transfer costs.
Parallel speedup is bounded by serial work and overhead; measure with realistic task sizes.