Chris Hager
Programming, Technology & More

Python, Threads and the Global Interpreter Lock (GIL)

This post is about Python, Threads, Multiprocessing, the GIL, and a great talk by David Beazley titled “Inside the Python GIL”.

Threads

Threads make it possible to execute multiple pieces of code in parallel, which means either utilizing multiple processors or having the operating system schedule execution time for the threads sequentially on one processor. In contrast to multiprocessing (forking) where multiple separated processes are started, all threads run in a single process and have access to the same resources. Python makes both threading and forking easy with the threading and multiprocessing modules.

Threads and the Python Interpreter

Since multiple threads access a single Python interpreter process with shared resources, Python uses a Global Interpreter Lock (GIL) to schedule access to the interpreter. Each thread has about 100 interpreter instructions before a context switch to the next thread occurs. As a result multiple threads in a single Python process cannot utilize more than one cpu, and cpu intensive code can actually run slower with threads than without. Only the multiprocessing module allows utilization of more cpu’s by starting multiple Python interpreter processes (with the downside of a more complicated message passing between the processes).

Inside the Python GIL

The 2009 teach talk “Inside the Python GIL” by David Beazley offers a lot of insight into how the GIL operates and is highly recommended for everyone interested in more details.

Slides: http://www.dabeaz.com/python/GIL.pdf



Share this post if you like it <3
swirl

Posted by Chris Hager in #python  ·  13 Oct 2010


swirl