Print
Python News

A Stackless PEP

08/23/2000

Gordon McMillan will be writing a Python Enhancement Proposal (PEP) for making the core Python interpreter stackless. A stackless interpreter for Python is an arcane concept. Maybe that is why it fascinates me. It appeals to the geek in me. Stackless is a project to untangle the Python interpreter from the C stack. C as in the C programming language; stack, as in a list onto which you push or pop things. Let me try to untangle that for you some.

The C stack tracks the execution of functions in a C program. When each new function is called, a pointer to the function and its state information is pushed onto the stack. When the new function terminates, the old function is popped off and the interpreter resumes processing it.

This is a pretty straightforward way of handling things for your average C program. Python, however, tracks this kind of execution itself in something it calls a Frame. Well, it almost does it itself -- it relies heavily on the C stack which maintains pointers to each Frame as it executes. This seemed like a good thing at the time. There are some problems with the current arrangement, though.

The C stack size is fixed; that limits the depth of functions you can call in a Python program. That can be a big issue when you are using recursion, a function calling itself repeatedly. Getting rid of the size limitation is one benefit of Stackless, but the real benefit comes when you want to manipulate the stack itself. When the Frame is decoupled from the C stack, you can do some tricky things, like hopping between functions on a stack and passing information between those functions, or maintaining multiple stacks and swapping them about. Stackless can be used for coroutines, a fast alternative to threaded programming; it makes state machines a cinch; and it makes it possible to use continuations, something Christian Tismer describes as "the most powerful control structure known in information science" and MacMillan refers to as "brain-exploding stuff." We will take a closer look at what you can do with Stackless in an upcoming series of brain-exploding articles.

Stackless Python is a bit of a misnomer. It still uses the C stack, it just separates Python's Frames from the C stack, making all these fancy tricks possible. It also isn't recent news. You can make Python stackless now. Christian Tismer maintains the stackless project and does what he can to keep his patches to make Python stackless current with the development of Python itself. Keeping up with Python development takes time, though. It's time Tismer could be putting into further enhancing Stackless Python. Many people have said Python's core should be stackless. If it is part of the core, then Tismer no longer has to play catch up. Gordon McMillan's PEP might actually make it happen.

Stackless may not be new, but PEPs are. They are a Python specific Request For Comments (RFC). They formalize the process of suggesting changes to Python. If you want to suggest a change, you write up your arguments and present them as a PEP. Those arguments are discussed, and eventually the PEP is either accepted or rejected. During the last round of calls for Python going stackless, the reply from the development team was very clear: "Write a PEP."

So that's what Gordon McMillan is going to do for Stackless. You can find an abstract for his PEP and many other PEPs on Python at Sourceforge.

If you are an avid user of Python, you should keep an eye on the PEPs. By commenting on them, you will help shape the future development of the language you use. The two PEPs that have inspired the most discussion recently have been PEP 211, which calls for new operators for performing linear algebra, and PEP 214, a suggestion for extending the standard print statement. Stackless is PEP 219 and is currently no more than an abstract. McMillan says it may take a few months to complete. To make including Stackless more compelling, the Stackless group wants to complete a simple API for creating coroutines. When the coroutines API and PEP draft are ready, it will either be a no-brainer to add Stackless to the core, or it will be the next big controversy.

Stephen Figgins administrates Linux servers for Sunflower Broadband, a cable company.


Read more Python News columns.

Discuss this article in the O'Reilly Network Python Forum.

Return to the Python DevCenter.