Although there are many factors you could attribute to Python's success, one is surely the ability to extend and embed Python in other languages. If there is anything Python itself can't do, but can be done from other languages, there is a good chance you can build a Python interface to this other language.
In this chapter, we discuss how to extend Python using the C/C++ and Delphi languages. The topics we cover include:
• Writing a Python extension in C (or compiling someone else's) by hand
• Using Simplified Wrapper and Interface Generator (SWIG) to make it easy to expose C and C++ libraries for use by Python
• Embedding a Python interpreter in a C++ application
• Dynamically calling DLLs without a C compiler
Delphi is another popular programming language that can operate at a low enough level to hook into Python at the C level, and a Delphi package exists to make this easier than the equivalent steps would be in C or C++. We also cover:
• Embedding a Python interpreter in a Delphi application
• Building a Python extension DLL in Delphi
When reading this chapter, you should keep in mind some capabilities offered by the Python COM framework, as we discussed in Chapter 12, Advanced Python and COM. When Python is using COM objects, you could view this as extending Python, since the capabilities offered by the COM object are made available to Python. When you create a COM object using Python, it's like embedding Python in another application: you make all the facilities of Python available to any application that uses your COM object.
Python is written in C and can be embedded in both C and C++ programs and extended with routines written in C and C++. The standard Python documentation includes a good tutorial on extending and embedding Python, which we won't repeat, and we certainly won't try to teach you C.
There are a number of reasons to use Visual C++ during your travels with Python:
• To build Python from sources. Many companies that ship Python binaries like to know they can build everything from C sources and have control over those sources (for example, checked into their source control system). The same applies to all Python extensions your application may use.
• To write a Python extensions as a DLL or if you have the sources to an interesting Python extension, but no binary for Windows for your Python version.
• To make an existing C or C++ library available to Python.
• To embed Python in a C or C++ application for one of the many good reasons for doing so!
 When you install Python in Windows, the C language header and library files are also optionally installed. These files are enough to build Python extensions on Windows without downloading the full Python sources. |
||||
Many of you are familiar with compiling C programs (and Python) on Unix, while others aren't C programmers at all. Although we attempt to make this chapter understandable for C novices, we certainly don't attempt to teach either the C++ language or the Microsoft Developers Studio environment.
It should be noted that once you start delving into building from sources, the world takes a murkier turn. When you download different source packages and attempt to build them into a single system, things rarely work as expected first time around.
One of the biggest problems is interproject dependencies. If you download the core Python sources, they will probably build well. However, if you then download some other Python extensions (such as the Win32 or COM sources), you may strike some problems when Visual C++ tries to find the Python files (such as header or library files) it needs.
The fundamental problem is that every programmer stores his or her source code in a different directory structure than everyone else. Sometimes these directory structures are dictated by corporate policy, or sometimes they are personal taste. Even worse than this is the problem of changing between versions of Python and other extensions you use. When a new version of Python comes out, you will probably need to change every single project file to reference the new version.
To attempt to make this problem bearable, here are a couple of tricks to make working with source code easier:
• Don't specify Python15.lib, PyWinTypes.lib, or PythonCOM.lib in your project file. These packages employ a Visual C++ trick so that by including the necessary headers (such as Python.h or PyWinTypes.h), an implicit reference is made to the necessary .lib files. This automatically uses the correct release (e.g., Python15.lib) or debug library (Python15_d.lib), and is also useful when upgrading Python; for example, Python 1.6 will presumably change the names of these .lib files, but this scheme avoids changing your project in that case.
• Use Microsoft Visual C++ to specify the Python and other directories globally, rather than on a per-project basis. This allows you to avoid referencing the location of the Python sources directly in your project file and applies for all projects. When you wish to move to a new version of Python, you can change Visual C++ so it references the new directories and rebuild all your projects.
The process of configuring Visual C++ in the manner described previously is quite simple. The first thing to do is to determine the location of the necessary Python files. If you installed the binary version of Python, these files are likely under the C:\Program Files\Python\Include and C:\Program Files\Python\libs. If you are building from the Python source archive, the directories for the headers may be something like Python-1.5.2\include and Python-1.5.2\pc, while the library files would be in Python-1.5.2\pcbuild. Then perform the following steps:
1. Start Microsoft Visual C++.
2. Select Tools 
Options Directories. Drop down the Show Directories For combo box, and select Include Files. Add the directory or directories where the Python headers can be located. Your screen should look something like Figure 22-1.
3. Select the combo box again, select Library files, and update the directories appropriately.
4. Select OK to close the dialog, and you're ready to go!
 |
||||
| Figure 22-1. Microsoft Visual C++ configured with the Python include directories |
||||
You may also wish to perform this same process for other common Python extensions you use. For example, you may make extensive use of the Win32 or COM extensions so you could perform this same procedure and specify their directories. Indeed, if you wish to build the COM extensions from sources you may need to ensure your directories are set up so that PyWinTypes.h and PyWinTypes.lib from the Win32 extensions can be found.
One of Python's most powerful features is how well it integrates with extensions written in C. If you have a library or toolkit available for C or C++, it is almost certain you can expose this library to Python using an extension module.
Alternatively, you may have a program with special requirements, such as crunching huge complex datasets, and your program could benefit immensely if you could speed up one small, but important part of the application. Writing this small speed-critical part in C and calling it from your Python program is an attractive solution many people adopt.
For whatever reason you need to do this, we provide here a short discussion of extension modules and building the modules on Windows.
For this example, we use a simple extension module from the Python tutorial. This creates a module called spam and exposes a function called system(), which runs a DOS command. This is obviously a contrived example as this same functionality can be obtained from the Python function os.system(); indeed, you would expect to find almost identical code implementing os.system(). The source code is as follows:
/* spammodule.c - pasted from Python extending/embedding manual*/
# include "Python.h"
static PyObject *SpamError;
static PyObject *spam_system(self, args)
PyObject *self;
PyObject *args;
{
char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return Py_BuildValue("i", sts);
}
static PyMethodDef SpamMethods[] = {
{"system", spam_system, METH_VARARGS},
{NULL, NULL} /* Sentinel */
};
#ifdef MS_WIN32
__declspec(dllexport)
#endif
void initspam()
{
PyObject *m, *d;
m = Py_InitModule("spam", SpamMethods);
d = PyModule_GetDict(m);
SpamError = PyErr_NewException("spam.error", NULL, NULL);
PyDict_SetItemString(d, "error", SpamError);
}
The format for Python extensions is well covered in the standard Python documents and in the various books to date. Our target is simply to get it to build on Windows. When this is done, you should end up with a file named spam.pyd, and be able to use it just like the following example:
>>> import spam
>>> spam.system('start notepad')
0
>>>
And you see a notepad appear on the screen.
Before we start on a fairly long spiel about how to create a Microsoft Visual C++ Project file by hand, we would be remiss not to make mention of a handy Python tool that can save you lots of heartache and may let you skip the next section completely!
Python on Unix has a well-established build procedure that uses a file named setup.in to control the build process. Each different operating system reads the setup.in file and generates an appropriate makefile for the system.
David Ascher has developed a tool that takes one of these Unix setup.in files, generates a Microsoft Visual C++ project file, then invokes Visual C++ to build the project. Although this would appear to be of interest mainly for people with existing Unix source code, these setup.in files are so simple that creating one of these files for our project is simpler than mastering Visual C++.
The simplest possible setup.in file consists of two lines. The first line contains *shared* (this tells Unix systems the module is to be built as a shared module, the equivalent of a DLL on Windows). The second line consists of the name of the module we wish to build, and the source files needed to build it. The setup.in file contains the following two lines:
*shared*
spam spammodule.c
Here, the module is called spam and there is a single source file, spammodule.c.
The tool that creates the Visual C++ project file is a single Python source file, compile.py, available from http://starship.python.net/crew/da/compile/. You can also find it, along with this book's code samples, at http://starship.python.net/crew/mhammond/ppw32/.
Using this tool to build the sample is simple; create a directory with spammodule.c and setup.in files, and from this directory, run the compile tool. A Visual C++ project is generated and the project built. Let's try this out. For this example, assume that compile.py is in your C:\Scripts directory and the spam files in C:\Scripts\spam:
C:\Scripts\spam>dir
Volume in drive C has no label.
Volume Serial Number is B423-50BF
Directory of C:\Scripts\spam
18/05/99 19:29 <DIR>
18/05/99 19:29 <DIR>
07/05/99 18:59 33 setup.in
07/05/99 14:53 761 spammodule.c
4 File(s) 794 bytes
113,135,104 bytes free
C:\Scripts\spam>..\compile.py
Attempting to start MSDev…
Building workspace (.dsw) file for workspace
Found python15.lib in L:\src\python-1.5.2\pcbuild
Found Python include directory in L:\src\python-1.5.2\include
Creating project (.dsp) file for spam
Building project
As you can see, you start with only the two files mentioned previously and run the compile tool. During this process, you should see Visual C++ start, the build process complete, then Visual C++ exit. Let's see what's in the directory now:
C:\Scripts\spam>dir
Volume in drive C has no label.
Volume Serial Number is B423-50BF
Directory of C:\Scripts\spam
18/05/99 19:31 <DIR> .
18/05/99 19:31 <DIR> ..
18/05/99 19:30 <DIR> pyds
07/05/99 18:59 33 setup.in
18/05/99 19:30 2,454 spam.dsp
18/05/99 19:30 1,182 spam.plg
07/05/99 14:53 761 spammodule.c
18/05/99 19:30 <DIR> tmp
18/05/99 19:30 618 workspace.dsw
18/05/99 19:31 33,792 workspace.ncb
18/05/99 19:31 49,664 workspace.opt
11 File(s) 88,504 bytes
112,688,128 bytes free
C:\Scripts\spam>dir pyds\ *.pyd
Volume in drive C has no label.
Volume Serial Number is B423-50BF
Directory of C:\Scripts\spam\pyds
18/05/99 19:30 20,480 spam.pyd
1 File(s) 20,480 bytes
113,736,704 bytes free
C:\Scripts\spam>
The script creates Visual C++ Project (spam.dsp) and Workspace (workspace.dsw) files (you can specify a different name for the workspace file by providing it as an argument to compile.py). There are some miscellaneous files created by the build process as well as two directories. The tmp directory is where the C object files (.obj) files are stored, and pyds is where the final Python extension is built. As you can see, the extension weighs in at around 20 KB.
All that's needed now is to copy the final spam.pyd to a directory on the PythonPath. C:\Program Files\Python\Dlls is where Python installed some standard extensions, so this may be suitable. Once spam.pyd is in a location where Python can find it, the sample code presented earlier should work.
There will be cases where you choose to use Visual C++ yourself rather than rely on the compile.py tool covered in the previous section. Although this takes longer to set up and doesn't provide a portable build solution, it does provide greater control over the build process. Of course, you're free to use both solutions: use the compile tool to generate your first project file and workspace, but maintain them manually after creation.
Either way, we will briefly discuss creating a project file from scratch using Visual C++ that builds our spam sample. Here are the three steps:
1. Create a new project file.
2. Modify the project settings.
3. Build the project.
Create a new project file. Creating a new project is fairly simple, although we're faced with a Microsoft Wizard! Complete the following steps:
1. Start Microsoft Visual C++.
2. Choose File New Projects Win32 Dynamic Link Library.
3. Under Location, choose a directory. We have used D:\MyStuff.
4. In the Project Name box, enter spam. A subdirectory called spam is suggested in the location box. The dialog should now look like Figure 22-2.
5. Select OK to start the New Project Wizard. When asked ''What kind of DLL would you like to create?'' choose "An empty DLL Project."
6. After a confirmation screen, VC++ creates an empty project, and the wizard exits.
7. Place the spammodule.c source file into the new spam directory.
8. Locate and select the File View tab at the bottom of the Workspace Explorer window.
9. Right-click on Spam Files, and select Add Files To Project from the context menu that appears and select the spammodule.c source file.
10. If you expand the Source Files tree and double-click on spammodule.c, it's opened in Visual C++, and the screen should now look like Figure 22-3.
Modify the project settings. We now have a project file, but some of the default settings aren't suitable for Python, so your next step is to modify some of the project settings. Specifically, you need to:
• Change the C runtime library settings so the extension and
Python share this library. This is a fairly obscure requirement, but
it's needed to prevent the
 |
||||
| Figure 22-2 Ready to create the spam project |
||||
extension module crashing under specific circumstances (see the later section "Debug Versus Release Builds" for more information).
• Change the extension of the final DLL to .pyd. Although unnecessary, this is an established convention that allows easy identification of Python extensions versus normal Windows DLLs.
• If you wish to perform a debug build, you need to change the debug version of the final DLL spam_d.pyd (again, see the later section "Debug Versus Release Builds" for more information).
The process to perform these steps is:
1. Choose Project Settings. In the top left combo box, select the Win32 Release configuration.
2. Select the C/C++ tab and drop down the Category combo box, stopping to admire the breathtaking lack of adherence to Microsoft's own GUI design standards. Choose Code Generation, and a new property page appears on the C/C++ tab. Change "Use runtime library" to "Multithreaded DLL." The property page should look like Figure 22-4.
3. Select the Link tab and change the name of the output file from Release/spam. dll to Release/spam.pyd, which should look like Figure 22-5.
 |
||||
| Figure 22-3. The completed project file |
||||
 |
||||
| Figure 22-4. Changing the C compiler options for the Win32 Release build |
||||
If you need to perform a debug build, repeat this process for the Win32 Debug configuration, but substitute Debug Multithreaded DLL for Multithreaded DLL in the first step and change the output file to spam_d.pyd as discussed previously.
 |
||||
| Figure 22-5. Changing the linker options for the Win32 Release build |
||||
|
||||
| You need the full Python sources to build a debug version of your extension, as Python15_d.dll isn't distributed with the Python binaries. Instead of building a full debug version, consider keeping the standard release build, but temporarily modifying the projects settings to include debug information. As discussed later in this chapter, as long as you ensure that the C runtime library in use is Multithreaded DLL, everything should work correctly. | ||||
Everyone works differently, so exactly how you manage your environment depends on your circumstances, but there are a number of other tips that relate to making your built extension module available to Python.
One alternative is to modify your PythonPath to explicitly include the directory where your PYD files are built; in our example, that would be D:\MyStuff\Spam\Release. Then the latest changes are available to Python as soon as the project is built, with no need to copy the extension anywhere else. A further refinement to this is to exploit the fact that release and debug builds create different files, spam.pyd and spam_d.pyd. This allows you to change your project to build our extensions into the same directory, avoiding the need to specify one directory for release builds and another for debug builds.
A second alternative is to exploit a feature in Visual C++ that allows you to execute an arbitrary command after your extension has built. It's quite common to use this feature to copy the final extension to another directory already on your PythonPath. To set this up, again go to the Visual C++ Project Settings, and you'll find, way off to the right in the available tabs, is Custom Build. Selecting this allows you to specify a simple copy command. Figure 22-6 shows the project setup to copy files into the C:\Program Files\Python\DLLs directory.
 |
||||
| Figure 22-6. Specifying a Custom Build step in Visual C++ |
||||
You're now almost ready to build, but faced with one Windows-specific complication the compile.py tools managed to hide. The problem is that you need to tell Windows that your module-initialization function (initspam()) is a public function, so Python can call it as the module is loaded. There are a number of ways to do this, the most common being: providing a module definition (.def) file, adding a special /EXPORT:initspam argument to the linker or modifying the code itself. For this demonstration let's take the last option and modify the module-initialization code from the original:
void initspam()
{
To:
#ifdef MS_WIN32__declspec (dllexport)
#endif
void initspam()
{
Figure 22-3 shows the source code after this change.
Building our project. Finally, we're ready to build the project. Perform the following steps:
1. Choose Build Set Active Configuration and select Win32 Release.
2. Choose Build Build spam.pyd. If all goes well, you see some messages in the Visual C++ output window, finishing like this:
---------------Configuration: spam - Win32 Release---------------
Compiling…
spammodule.c
Linking…
Creating library Release/spam.lib and object Release/spam.exp
Performing Custom Build Step on .\Release\spam.pyd
1 file(s) copied
spam.pyd - 0 error(s), 0 warning(s)
Note that because you specified an additional copy operation after the build, you can see the 1 file(s) copied message generated by the copy. A quick check in C:\Program Files\Python\DLLs shows the new spam.pyd.
Now, start Python (or PythonWin). Type import spam, followed by spam.system (‘start notepad’) or the command of your choice.
A couple of times now, we have glossed over some differences between debug and release builds for our extensions and Python. This is a symptom of the important, although somewhat technical, issue of the C runtime library.
If you don't know what a C runtime library is or don't care about the technical details, the simple rules to follow are:
• Release builds of your project must use the Multithreaded DLL and link with the standard Python .lib files.
• Debug builds of your project must use the Debug Multithreaded DLL C runtime library, must link with the _d version of the Python .lib files, and must themselves be named following the _d convention.
This is simple to set up, as we demonstrated when building the spam sample. The compile tool described previously automatically creates the correct settings, so in some cases you don't need to do anything. However, a deeper understanding of the issues will help you understand why the _d convention exists and how to exploit or work around it for your situation. Feel free to skip the rest of this section.
The underlying issue is that Python and its extensions are DLLs, and these DLLs need the same C runtime library. Particularly at issue are FILE objects and memory allocated via malloc(). If all Python extensions aren't using the same C runtime library, the FILE objects passed between Python and the extensions are considered invalid. The result is likely to be an access violation.
Although this problem isn't unique to Windows, Microsoft Visual C++ is one of the few compilers that provide different libraries for debug and release builds. The debug libraries are useful and contain all sorts of diagnostic and sanity checks for using these objects. As a result, it's common for a programmer to build a debug version of his module, but attempt to run it with a release version of Python or other modules. The result is a problem far worse than he was originally trying to debug, so Python invented a scheme to avoid this common pitfall.
The idea is that when Python itself is built for debug, it looks for only modules with _d appended to the name. A debug build of Python then finds only specific debug modules, and a release build of Python finds only release modules.
As you can see, the _d convention is not a true solution to the problem; the underlying issue is that Python and all the extensions must be built with the same, shared C runtime library. Python's use of _d is a convention that makes the more common errors less likely. This convention also means it's possible to have both debug and release builds of the same extension in the same directory. When you run Python.exe, the release extensions are automatically found. When you run Python_d.exe, the debug extensions are found without needing to change the PythonPath. This makes debugging quite simple: just run Python_d.exe under the debugger and your debug extension module can be debugged automatically.
The biggest problem with this scheme is that to debug a single extension module, you must have debug builds of every extension module your program uses. Depending on the application, this may be difficult and even impossible. You can exploit this information to make your debugging easier. Because the issue we have been discussing is the C runtime library, you can change almost every other compiler or linker option for your debug or release build except this. Thus, for your release build, you can temporarily disable optimizations and enable symbolic debugging information; just make sure the C runtime library never changes. Then step through your extension in the C debugger (but not through Python or other extensions) before restoring the default project settings.
There are occasions when it's necessary or desirable to build Python itself from its sources. As mentioned previously, reasons may include wanting to build a debug version, or simply the satisfaction or security of knowing you can build your entire project from scratch.
The process is quite simple, and Python builds easily. The sources come with Visual C++ project and workspace files, and building is as simple as opening the workspace and starting the build.
The sources are available from http://www.python.org, usually in a file with the name pythxxx.tgz, where xxx is the version of Python. For example, Python 1.5.2 sources are available in pyth 152.tgz. The .tgz file is a gzipped tar file and is understood by the ubiquitous WinZip program (http://www.winzip.com). Once you expand this archive into a suitable directory (be careful to maintain the directory structure when expanding), you should have a Python-1.5.2 directory with a number of subdirectories including PCBuild, Python, Modules, and so forth.
In the PCBuild directory you'll find pcbuild.dsw; this is the Visual C++ workspace. Once opened, Visual C++ should look similar to Figure 22-7.
 |
||||
| Figure 22-7. The Python sources ready to build |
||||
Depending on your requirements, you may wish to build some or all of the projects. At a minimum, you need to build the Python15 project that builds Python15.dll (or Python15_d.dll for debug builds). All built files are placed in the PCBuild directory, ready to be distributed, copied to the Windows System directory, etc.
Visual C++ generates large numbers of temporary files, often totaling megabytes. When you have created a DLL or EXE you are happy with, you may choose to copy it somewhere else and choose Build Clean. This deletes all the built files reclaiming the list disk space.
The process we described previously of building a DLL extension module for Python is known as extending Python; you extend its capabilities with a new module. The other common process is to put Python inside another existing application, possibly for use as a macro or extension language. This process is known as embedding Python.
If you look at the files Python installs, notice that the bulk of Python itself is implemented in a DLL, Python15.dll, for all Python 1.5.x versions. Python.exe itself is tiny, and simply uses the Python DLL. In fact, consider Python.exe as a trivial example of how to embed Python in your application.
Python exposes a large API that embedded programs use. This API is large and rich; almost anything you can do from Python code you can do from the C API, including running code, calling objects, creating new objects, pulling apart Python objects (such as getting the string value from a Python string object), and so forth.
The best way to get started with embedding Python is to read the excellent extending and embedding documentation (optionally installed with Python) in conjunction with the sample in the Demo\Embed directory of the Python sources. You can then peruse the Python/C API documentation (also optionally installed with Python).
SWIG (available at www.swig.org) is a tool written by David Beazley that helps connect programs written in high-level languages such as Python, Perl, and Tcl/Tk with low-level C and C++ code. It's a substantial package with a large following. Rather than having to write your own wrapper code to intermediate between Python and C data structures, SWIG automates the process of producing a Python extension based on a description of the functions you wish to export. A large number of popular Python extensions have been built using SWIG, including parts of the Win32 extensions. You should look into SWIG before embarking on any nontrivial Python extension project.
Space doesn't permit a detailed example of SWIG, but we refer you to the excellent documentation that comes with the package. This covers all the specifics relating to building Python extensions under Visual Studio.
The Big Three development environments must be Visual Basic, Visual C++, and Borland Delphi. Delphi users feel that the language and development environment combine the ease of use of Visual Basic with the power of C++, and the Visual Component Library that comes with Delphi does a superb job of hiding the complexities of the Windows API. Delphi is particularly popular in Europe, where Pascal has often been the teaching language for computer science.
Delphi uses Object Pascal, the latest stage in the evolution of Pascal. Like C++, Delphi is a fully compiled language that allows both high-level object-oriented programming and "down-to-the-metal" manipulation of raw memory and pointers. It's a pure Windows tool that allows you to write DLLs and EXEs and to access all the Windows API calls. It also supports the C calling convention. This means that in theory Delphi can access the main Python DLL in the same way as C/C++ programs, and can compile DLLs Python can use as extensions.
During 1997 and 1998, a package called PyDelphi (included in the examples for this chapter at http://starship.python.net/crew/mhammond/ppw32/ and also available at www.multimania.com/marat/delphi/index.htm) evolved through a merger of work by Morgan Martinet, Dietmar Budelsky, and Grzegorz Makarewicz. This makes integration of Python and Delphi at the C level almost effortless. PyDelphi includes a Delphi package library that can be installed in the component gallery and a comprehensive set of demos and tutorials.
The core component is a Pascal module called PythonEngine.pas, which declares Pascal types to match those in Python.h and function headers to match most of those exported by the Python DLL. This makes it possible to access Python the same way as C/C++. Going on from this, the authors provide a number of components that represent the Python engine itself, Python modules and variables, and I/O components.
Figure 22-8 shows the first example from the PyDelphi tutorial.
To achieve this, you create a new project with the visual components shown in Figure 22-8. In addition, place two special Python-Delphi components on the form. These are invisible at runtime, but allow you to set up many properties from the Object Inspector rather than in code. The essential one is a PythonEngine, (highlighted in Figure 22-9).
The PythonEngine component has events that can be trapped for initialization and finalization, and—most usefully—an InitScript property of type TStrings. This is a list of strings that can be set at design time in an editor and can hold any Python code you want.
 |
||||
| Figure 22-8. Python interpreter inside a Delphi application |
||||
 |
||||
| Figure 22-9. The same form at design time |
||||
In addition, you add a PythonGUIInputOutput component. After setting a couple of properties, this redirects the output of the Python engine to the upper Rich Edit window in the form. The user can now type Python code into the lower window and execute it with a click on a button.
In addition to the usual header, you need one line of code behind the button:
procedure TForml.Button1Click(Sender: TObject);
begin
PythonEngine1.ExecStrings( Memo1.Lines );
end;
PyDelphi exposes almost the entire Python C API; but Delphi also has a very useful data structure called a StringList, and PythonEngine has some higher-level functions to work with these, such as ExecStrings, in the previous snippet. A related and useful trick with PyDelphi is to place invisible list boxes and memos on a form and paste chunks of Python, or even whole modules, into them; these can easily be passed to Python later. To distribute an application, you need to include just the Delphi executable and the usual python15.dll, plus any extra Python modules required; these can be frozen inside the Delphi executable if desired.
It's easy to create a DLL in Delphi and expose it as a Python module. Let's start with one of the PyDelphi tutorial examples, then extend it into something more useful. The initial example exports one function to add two numbers together. In Delphi, choose File New DLL to create a DLL project, and save it. Then create a new Pascal module (called just module.pas) containing the following code:
unit module;
interface
uses PythonEngine;
procedure initdemodll; cdecl;
var
gEngine : TPythonEngine;
gModule : TPythonModule;
implementation
function Add( Self, Args : PPyObject ) : PPyObject; far; cdecl;
var
a, b : Integer;
begin
with GetPythonEngine do
begin
if PyArg_ParseTuple( args, 'ii:Add', [@a, @b] ) <> 0 then
begin
Result := PyInt_FromLong(a + b);
end
else
Result := nil;
end;
end;
procedure initdemodll;
begin
try
gEngine := TPythonEngine.Create(nil);
gEngine.AutoFinalize := False;
gEngine.LoadDll;
gModule := TPythonModule.Create(nil);
gModule.Engine := gEngine;
gModule.ModuleName := 'demodll';
gModule.AddMethod( 'add', @Add, 'add(a,b) -> a+b' );
gModule.Initialize;
except
end;
end;
initialization
finalization
gEngine.Free;
gModule.Free;
frmAbout.Free;
end.
You can see the similarity between this and the minimal C extension earlier in the chapter. However, PyDelphi includes a TPythonModule component that slightly changes the initialization of the Python function names. Having written this, you can edit the Delphi project file (extension DPR) to export the initdemodll function:
library DemoDll;
uses
SysUtils,
Classes,
module in 'module.pas';
exports
initdemodll;
{$E pyd}
begin
end.
The project can be compiled to a DLL and saved with the extension .PYD somewhere on the PythonPath.
If you are creating a Python script to be run from a command prompt, it's perfectly feasible to write GUI code in Delphi. The DLL could contain forms and dialogs, which can be launched from within a DLL function called by Python. One use for this might be to provide stock dialogs for a Python script, although you could go further and write an entire application that is launched from Python and uses a Delphi GUL.
For reasons discussed in Chapter 20, GUI Development, don't try to run DLLs that use a GUI within the PythonWin editor. Plain-vanilla Object Pascal extensions are safe to use in any Python IDE, but the Delphi VCL application framework and the MFC application framework in PythonWin will both assume they are running the application and so, you get an immediate error message.
The authors of the package have also written their own Python IDE in Delphi and a tool that automatically examines Pascal source code and generates Python wrappers for the entire Visual Component Library. These make it possible to write Python code that creates Delphi forms and objects and accesses their methods and properties, just as you would in Delphi. The following is valid Python code:
from Forms import *
from StdCtrls import *
f = TForm().CreateNew(Application)
Unfortunately, this incredible capability is still experimental and limited to use within Delphi applications; thus, you can write a Delphi application with an embedded Python macro language that can create and manipulate Delphi GUI elements, but you can't just write a plain Python script. The latter is technically feasible and it is a development we hope to see in the future.
As we mentioned in Chapter 13, Databases, Sam Rushing (http://www.nightmare.com) has written an extension that allows Python to dynamically load and call any function in any DLL, not just special Python extensions. For the C programmers among you, these are equivalent to using the Windows API functions LoadLibrary() and GetProcAddress().
These tools offer a great deal of power but also carry risks. When using Python extension modules such as the Win32 extensions, you can be fairly confident the worst that will happen is a Python exception. When dynamically accessing a DLL, you are responsible for ensuring the arguments you pass are the correct type; an error in doing so can corrupt the stack and lead to either seriously misleading results or a crash.
Using these tools, Sam has built a comprehensive ODBC module that allows access to every function in the ODBC API and to a GUI application framework sitting directly on top of the Windows API.
In general, the safest way to access existing C libraries is to build Python extension modules. However, there may be occasions when a C compiler isn't available or when dynamic loading and unloading are required. A case study mentioned in Chapter 1, What Is Python?, uses Calldll/Windll to interface to an encoding translation library that moves megabytes of data between operating systems each day; for political reasons, C/C++ development was not an option in this case.
Two packages are available from Sam Rushing's site. CallDLL consists of a Python extension module, calldll.pyd that exposes a low-level API for loading modules and calling functions within them. The DynWin package builds on this to offer a much easier high-level API, and includes the GUI library mentioned previously. The package includes a key module, windll.py that provides a higher-level wrapper around CallDLL making it easy to use. In fact, only two files are needed for dynamic loading: calldll.pyd, and windll.py, both of which should be installed on the PythonPath. These are included with the examples for the chapter, which can be found at http://starship.python.net/crew/mhammond/ppw32/.
As an example we've built an extremely basic DLL called simple.dll. This exports two functions, Min (a, b) and Max(a, b), which return the minimum and maximum of two numbers. All arguments and return types are integers. The following example shows how to load and use the DLL, presuming it's in c:\temp. You can omit a path if it's on the Windows path:
>>> from dynwin.windll import *
>>> mod1 = module('c:\\temp\\simple') # loads the DLL
>>> mod1.handle # it can report its location in memory
22806528
>>> mod1.Min(27, 28) # loads and executes Min function
27
>>> mod1.Min # we now have a 'callable function' object…
<callable function "Min">
>>> mod1.Min.address #…which knows its address too
22836704
WinDLL is doing a lot of work behind the scenes here, using Python's abilities to introspect and trap attribute access. Go to windll.pys source to see how it works.
WinDLL can transparently handle any integer or pointer arguments. The vast majority of Windows API calls have arguments that are either an integer, a pointer to a string, or some other structure, all of which require four bytes of memory. If you need to handle other types of arguments, it may be necessary to drop down a level and use the lower-level argument-formatting functions in CallDLL.
WinDLL also includes a class cstring that makes it easy to pass string or character buffer arguments back and forth. This class should be initialized with a Python string (which may be empty), and an optional length. Internally, cString maintains a buffer with a null-terminated string and the address of the buffer it passes to the DLL when used as an argument. To test this, the DLL exports a function StringRepeat that repeats a string a number of times:
>>> inBuf = cstring('spam') # make a buffer holding a c string
>>> outBuf = cstring(",50) # make another big enough for output
>>> mod1.StringRepeat(inBuf, outBuf, 10) # returns the length of out string
40
>>> outBuf
'spamspamspamspamspamspamspamspamspamspam'
Rather than go any further, you can refer to the documentation and examples in the CallDLL and DynWin packages.
Extending and Embedding the Python Interpreter, by Guido van Rossum is included in HTML format with every Python distribution. Just click Help on the PythonWin toolbar.
O'Reilly's Programming Python and IDG's-Internet Programming with Python, by Watters, van Rossum and Ahlstrom, both contain sections on extending and embedding. The former is far more detailed while the latter is ideal starting point for beginning C programmers.
SWIG lives at http://www.swig.org/ and has good manuals available for download.
PyDelphi is available from http://www.multimania.com/marat/. You can also find it, along with this book's examples, at http://starship.python.net/crew/mhammond/ppw32/.
CallDLL and WinDLL are available in the DynWin package from http://www.nightmare.com. The key files are included with the examples at http://starship.python.net/crew/mhammond/ppw32/.
This chapter provided a brief introduction to extending Python's capabilities using Visual C++ and Delphi languages. The topics included writing Python extensions and extension DLLs, embedding a Python interpreter with a C++ or Delphi application, and exposing C and C++ libraries for use by Python.