by

Binding to dynamic libraries from your Delphi applications.

During one of my webinars this morning on migrating services from windows to Linux, a question was raised about how to import libraries to Delphi. In this post I’d like to explain what I’d done in that webinar in more detail, and show you how you can import code from external libraries into your Delphi applications.

In this post, as in the webinar, I’m referencing code from my earlier blog post on launching external applications under linux: https://chapmanworld.com/calling-linux-commands-from-delphi/

Dynamic Libraries.

All of the operating systems supported by Delphi have some form of dynamically loadable library.

  • DLL (dynamic link library) on MS-Windows.
  • Dynlib (dynamic library) on OSX
  • SO (shared object) under Linux / Android and iOS

In each case, they serve essentially the same purpose. These are libraries of code which may be loaded at run-time and provide access to some additional functionality.

In most cases to the operating system API will be exposed through these libraries. For example, under MS-Windows we have the Kernel32.dll and User32.dll libraries which provide the bulk of the Win32 api. Similarly under Linux operating systems the libc.so library offers access to operating system API features.

Dynamic Linking

As I said above, external libraries are loaded at run-time. This means that they are not statically linked to your application when it is built, but rather, they are loaded into your applications memory space as it runs. This is achieved by calling the OS API and requesting that it load the library, provide function pointers to the required functions, and ultimately unload the library when the application ends.

The Delphi compiler offers two mechanisms for loading dynamic libraries, each of which does the very same thing under the hood, but which affect the time at which the external library is loaded by your application. In the absence of good descriptive terminology for the two mechanisms, I’ll refer to them as Early Loading and Late Loading.

The first of these mechanisms, Early Loading, is to declare a prototype for the external library function in your code, which I’ll show you how to do shortly. This mechanism causes the compiler to inject the necessary code to load, bind to, and unload the library automatically. This means that the external library function is available from the very moment your application starts up. When using this mechanism, if the OS is unable to find the library, or if the function you are trying to load is not found within that library, an error (exception or otherwise) will be raised as your application starts up.

The second of these mechanisms, Late Loading, is that you declare the function as a function pointer, and manually request that the OS load your library, provide the function pointer, and unload your library when you’re done with it. This gives you more control over when the library is loaded, and can be useful in cases where the path to the library is unknown at the time of writing the application. Many plug-in systems function this way, for example, this is how Microsofts IIS web server loads ISAPI modules, which are simply DLL files.

In this post I’ll be showing you how to translate a function header for the Early Loading mechanism. In a later post I’ll follow up with an example of Late Loading.

As a side note, libraries may also be statically linked to your application using the first of the above mechanisms. When a library is statically linked to your application, that library is essentially copied into the executable file and is therefore always available, and does not require calls to the OS to load it. Static libraries however, are a subject for another blog post, and it is usually not possible to bind a library built for dynamic loading statically. For the remainder of this post, I’ll discard anything pertaining to static linking, and focus only on dynamic linking.

Library Headers

Most C/C++ developers will be familiar with header files. Essentially a header file contains a series of function prototypes for functions which are in external libraries.This is the C/C++ equivalent of the first loading mechanism that I described above. The compiler will load the dynamic library and acquire the function pointers automatically when the application starts up.

Since C/C++ are the languages used to write the majority of popular operating systems, and therefore also their API libraries and the majority of third party libraries, header files are usually provided for these libraries in the C/C++ languages. This poses the first problem for us Delphi developers wanting to use dynamic libraries, we must first obtain or write translations of the header files in pascal.

Often, if the library you’d like to use is very popular, someone else will already have written the header translation and made it available online. This is the case for example with the OpenGL and DirectX libraries. In order to use these libraries, you simply need to download the appropriate header files. In many cases however, such translations aren’t readily available, and you’ll be in the position of writing your own translation.

There are some tools available which can parse C/C++ headers and generate the pascal equivalent for Delphi. These tools however, have varied capabilities, and rarely do a complete or even a ‘good’ job of translating the headers. Instead, though it involves a little more work, it’s usually beneficial to manually port the functions that you need. Though doing so may/will require at least some knowledge of C syntax in order that you can write the equivalent function prototype.

Example translation

In my recent webinar, I manually translated three function prototypes from the libc libarary, on of the Linux API libraries, into pascal. For C&C++ compilers the prototypes for libc are divided among several header files, the functions which I translated are part of the stdio.h header file. In this case however, I did not translate the header it’s self, but rather, I grabbed the function prototypes that I needed from online documentation.

The three functions are popen(), pclose(), and fgets(). The online documentation for these three functions can be found at man7.org…

The documentation for each of these functions contains the C/C++ function prototype. Lets take a look at the fgets() function first, since it’s actually the simplest to translate. Here are the C and Pascal versions of the function prototype.


char *fgets(char *s, int size, FILE *stream);

function fgets(buffer: pointer; size: int32; Stream: TStreamHandle): pointer; cdecl; external libc name _PU + 'fgets';

So let me walk you through how I wrote the pascal prototype. Some of this depends on prior C++ knowledge, however, I’m sure that if you have no C/C++ experience you could still come to the same conclusion with little more than an aptitude for using a popular search engine.

First of all, I need to decide if this is a function or a procedure. The C prototype begins ‘char *’ which is the return type. As this is not ‘void’, we have a function. ‘char’ indicates a character type, and the asterisk indicates that this is a pointer, so the natural choice for pascal would be…

function :pchar;

or

function :pansichar;

However, there are two stumbles here. First of all, there are no pchar / pansichar types in pascal when targeting Linux. Such types would assume ANSI characters, which is not necessarily the case, and as it turns out (targeting Ubuntu 16.04) I actually get UTF8 data back from a call to fgets. I decided the best way to handle this was to simply use the pointer datatype, and worry about translating the data elsewhere in my project, and so the function prototype becomes…

function :pointer;

Okay, so the next part of the C prototype is the name of the function, fgets, that’s easy enough…

function fgets():pointer;

And now we’re on to the parameters being passed into the function. The first parameter is another ‘char *’ named s. Well, I want to be consistent, so lets use the pointer type again.

function fgets( s: pointer ): pointer;

The second parameter is an ‘int’ integer type parameter named size. As it happens the ‘int’ datatype in C/C++ is a 32-bit signed integer, so I could use the pascal types ‘integer’ or ‘int32’. Well, int32 contains more information in the type name, in particular that it is 32-bits wide, so I’ll use that type (since the integer type does not specify it’s size, it could change in later revisions of the compiler, or other pascal compilers).

function fgets( s: pointer; size: int32 ): pointer;

The final parameter is of type ‘FILE *’ which in C is a pointer to a file, where FILE is actually a handle. In software API’s it is rare that you ever want to (or should) manipulate a handle directly, instead, a handle is something which you would obtain from one API function and pass to successive API calls. Since we don’t need to manipulate it, it doesn’t actually matter what data type FILE is, all that matters is that we’ll be passing a pointer to it. So the third parameter becomes a pointer also.

I decided to give this third parameter it’s own data type ‘TStreamHandle’ because I can now use this data type anywhere I come across ‘FILE *’ when translating other functions in the same library.

type
TStreamHandle = pointer;

function fgets( s: pointer; size: int32; stream: TStreamHandle ): pointer;

So that’s the C prototype translated, but there is some other information (contained elsewhere in the C header) which we need to add for the Delphi compiler.

First of all, we need to specify a calling convention for this function. Under windows the default calling convention for dynamic libraries is ‘stdcall’ which is a variant of the ‘pascal’ calling convention, however, for just about every other platform the default is ‘cdecl’.

function fgets( s: pointer; size: int32; stream: TStreamHandle ): pointer; cdecl;

Now we need to tell Delphi that this function is implemented in an external library, and tell it which library that is. Note that when your application starts up, it will search first in the same directory as the application, and then in directories on the search path for your library. In the case of libc on Linux, this library will always be on the search path…

function fgets( s: pointer; size: int32; stream: TStreamHandle ): pointer; cdecl; external libc;

Finally, the name of the fgets function in the external library could be different to the name we’ve given to our function prototype. For example, we could have named this function fgetstring(), but specified that it is a prototype for the external ‘fgets’ function. So we need to specify the external function name.

function fgets( s: pointer; size: int32; stream: TStreamHandle ): pointer; cdecl; external libc name 'fgets';

(As a side note, in my example I used name _PU+’fgets’ because on some operating systems external function names are prefixed with an underscore, the Delphi compiler provides the _PU constant to assist with cross platform support, but it is not specifically necessary for Linux-only code)

There we have it, the fgets function is now available to our pascal code. Lets take a quick look at the other two functions that I imported for my webinar code..

///  <summary>
///    Man Page: http://man7.org/linux/man-pages/man3/popen.3.html
///  </summary>
function popen(const command: MarshaledAString; const _type: MarshaledAString): TStreamHandle; cdecl; external libc name _PU + 'popen';

///  <summary>
///    Man Page: http://man7.org/linux/man-pages/man3/pclose.3p.html
///  </summary>
function pclose(filehandle: TStreamHandle): int32; cdecl; external libc name _PU + 'pclose';

Translating the pclose() function is quite obvious given the translation of fgets(), as it uses the same data types. popen however contains two things not seen in the fgets() translation.

First, I prefixed the ‘type’ parameter name with an underscore. This is because ‘type’ is a reserved word in pascal, but is used as the parameter name for the C header. It’s fine to rename parameters when translating a header, so long as the data types match it should all work out fine.

Second, I’m using the MarshaledAString data type for the ‘command’ and ‘_type’ parameters, which in the C header each appear as

const char *

Essentially the function is expecting another pchar,  but simply using ‘pointer’ as the data type would mean translating a string to a pchar for passing into the function. Delphi provides the MarshaledAString data type which allows me to pass regular strings for these parameters, which is just too convenient to pass up on!

Conclusion

First of all, an apology. I like to write posts which stand-alone, but this post is something of a follow up to an earlier one on launching external applications under Linux. For a working code sample, please refer back to that post, and see the comments attached to it.

The potential offered by binding to dynamic libraries is very powerful, however, I feel it carries more importance for Linux at this time. You see, most of the OS API’s for windows are already translated and form the underpinnings of the VCL, and so you rarely need go to such low levels.

Since the introduction of the Linux platform to Delphi, being absent of VCL and even FMX without a third party add-on, there’s a weft of capabilities in the platform for which you simply have to bind for yourself. This is done through the use of external dynamic libraries. Go forth and translate headers, and consider sharing your translations for others in the Delphi community!

Thanks for Reading.