We've spent a bit of time working with the raw Python API, so let's now write a C++ wrapper library to make using Python a bit easier.

Initialization and shutdown

The first thing we'll take care of is initializing Python, and cleaning it up.

Programs must call openPython() when they start up, to initialize Python. I've made threading optional, but if you know you will always want it (or not), you can, of course, just take this flag out.

static bool gIsOpen = false ;
static PyThreadState* gpMainThreadState = NULL ;

void 
openPython( bool enableThreads )
{
    assert( !gIsOpen ) ;

    // initialize Python
    Py_Initialize() ;
    gIsOpen = true ;

    // initialize threading 
    if ( enableThreads )
    {
        PyEval_InitThreads() ; // nb: creates and locks the GIL

        // NOTE: We save the current thread state, and restore it when we unload,
        // so that we can clean up properly.
        gpMainThreadState = PyEval_SaveThread() ; // nb: this also releases the GIL
        assert( gpMainThreadState != NULL ) ;
    }
}

Programs should also call closePython() when they shutdown.

void 
closePython()
{
    assert( gIsOpen ) ;

    // clean up threading
    if ( gpMainThreadState != NULL )
    {
        PyEval_RestoreThread( gpMainThreadState ) ; // nb: this also locks the GIL
        gpMainThreadState = NULL ;
    }

    // clean up Python
    Py_Finalize() ;
    gIsOpen = false ;
}

Exceptions

The next thing we'll write is an exception class that represents a Python error. Whenever a Python API call returns an error status, we will retrieve the details of the error, then convert it to a C++ exception.

class Exception : public std::runtime_error
{
public: // constructors/destructor
    Exception( const char* pExceptionMsg ) ;
protected: // constructors
    Exception( const std::string& errorMsg , 
               const std::string& excType , const std::string& excValue , const std::string& excTraceback 
             ) ;
public: // miscellaneous methods
    static void translateException() ;
private: // data members:
    std::string mExcType ;
    std::string mExcValue ;
    std::string mExcTraceback ;
} ;

The class carries the 3 pieces of Python error information, the type, value and traceback. We also allow an exception to be thrown with an arbitrary message, in case we detect an error ourself^[1]In which case, there won't be the normal type/value/traceback values available..

The important part of this class is how it translates a Python error into a C++ exception.

void
Exception::translateException()
{
    // get the Python error details
    string excType , excValue , excTraceback ;
    PyObject *pExcType , *pExcValue , *pExcTraceback ;
    PyErr_Fetch( &pExcType , &pExcValue , &pExcTraceback ) ;
    if ( pExcType != NULL )
    {
        Object obj( pExcType , true ) ;
        auto_ptr<Object> attrObj( obj.getAttr( "__name__" ) ) ;
        excType = attrObj->reprVal() ;
    }
    if ( pExcValue != NULL )
    {
        Object obj( pExcValue , true ) ;
        excValue = obj.reprVal() ;
    }
    if ( pExcTraceback != NULL )
    {
        Object obj( pExcTraceback , true ) ;
        excTraceback = obj.reprVal() ;
    }

    // translate the error into a C++ exception
    stringstream buf ;
    buf << (excType.empty() ? "???" : excType) ;
    if ( ! excValue.empty() )
        buf << ": " << excValue ;
    throw Exception( buf.str() , excType , excValue , excTraceback ) ;
}

Whenever a Python API call returns an error, we simply call Exception::translateException() to get the error details and throw a C++ exception.

Managing interpreters

Next we'll set up a class to manage interpreters^[2]Note that this is only needed if you are using Python from multiple threads..

class Interpreter
{
public: // constructors/destructor
    Interpreter() ;
    virtual ~Interpreter() ;
public: // interpreter methods:
    void lockInterpreter() ;
    void unlockInterpreter() ;
private: // data members
    PyThreadState* mpThreadState ;
} ;

When we create a new Interpreter object, we ask Python to allocate an interpreter for us.

Interpreter::Interpreter()
{
    // create a new interpreter 
    PyEval_AcquireLock() ; // nb: get the GIL
    mpThreadState = Py_NewInterpreter() ;
    if ( mpThreadState == NULL )
        Exception::translateException() ;
    PyEval_ReleaseThread( mpThreadState ) ; // nb: this also releases the GIL
}

Note the call to Exception::translateException() if Python fails to allocate a new interpreter.

When we're done using the Interpreter object, we have to clean things up.

Interpreter::~Interpreter()
{
    // clean up
    PyEval_AcquireThread( mpThreadState ) ; // nb: this also locks the GIL
    Py_EndInterpreter( mpThreadState ) ;
    PyEval_ReleaseLock() ; // nb: release the GIL
}

We also need methods to swap an Interpreter in (when we want to use it) and out (when we're done using it).

void Interpreter::lockInterpreter() { PyEval_AcquireThread( mpThreadState ) ; }
void Interpreter::unlockInterpreter() { PyEval_ReleaseThread( mpThreadState ) ; }

We also set up a helper class to manage the process of locking and unlocking Interpreter's.

class InterpreterLock
{
public:
    InterpreterLock( Interpreter& interp ) ;
    ~InterpreterLock() ;
private:
    Interpreter& mrInterpreter ;
} ;

This uses the RAII pattern, where the Interpreter is locked when the InterpreterLock object is created, and unlocked when it is destroyed. This means that the Interpreter object will always get unlocked when the InterpreterLock object goes out of scope, even if an exception is thrown, or the code exits prematurely.

We also define a few macros that let us write code like this:

BEGIN_PYTHON_INTERPRETER( interp )
{
    ... // do stuff with the interpreter here
}
END_PYTHON_INTERPRETER()

This makes it easy to see what's going on - the Interpreter object gets locked when the block is entered, and unlocked when the block is left^[3]Regardless of how the block is left e.g. running off the end, an exception thrown, a return or goto statement..

Managing objects

Finally, we need a class to wrap the many PyObject's we'll be dealing with.

class Object
{
public: // constructors/destructor
    Object( _object* pPyObject , bool decRef ) ;
    virtual ~Object() ;
public: // miscellanous methods
    PyObject* pPyObject() const ;
private: // data members
    PyObject* mpPyObject ; // Python object being wrapped
    bool mDecRef ; // flags if we should decref 
} ;

The important thing to note here is the decRef parameter on the constructor. This flags whether we own the reference on the object, and therefore must decrement the reference count when the Object is destroyed.

Object::Object( PyObject* pPyObject , bool decRef )
    : mpPyObject(pPyObject) , mDecRef(decRef)
{
}

Object::~Object()
{
    // clean up
    if ( mDecRef && mpPyObject != NULL )
        Py_DecRef( mpPyObject ) ;
}

While most of the time, we will want to decrement the reference count, I didn't put a default value on the parameter, since this forces you to think about whether it needs to be done or not.

We've now got all the scaffolding done; in the next tutorial we'll take a look at the fun stuff and get Python actually doing things.

Download the source code here.

Next: Writing a C++ wrapper library (continued)