It Seemed Like A Good Idea, At The Time

The largest obstacle in my path when I try to accomplish anything in scheme is the lack of a standardized foreign function interface (FFI).  I understand the reasons for it, I understand that there are many implementations running on platforms and architectures where a foreign function interface wouldn’t make any sense, I understand that in some cases, such as JVM-based implementations, any usable FFI would have to look radically different from the standard C library interface that we most often want.  I understand all of this, but still it annoys me a little.

My needs in an FFI are simple.  All I want to do is to load C libraries such as OpenGL and GTK, and call the functions they export.  I don’t need anything fancy, like callbacks or continuations.  I just want something that lets me make stuff appear on the screen (the ability to use optimized math libraries from within an interpreted language would be a plus, too).

Most implementations offer a C FFI or some sort, but their usage is always radically different.  They have different capabilities, and different failure conditions, and even if their mode of operation is the same, the command semantics are usually different.  Once you start using one of them, you aren’t coding in scheme any more, you’re coding in PLT Scheme, or Gambit Scheme, or Chicken Scheme, or any of the myriad other implementations out there.  And this is ignoring all those implementations, such as Ikarus, who for the sake of simplicity don’t even offer an FFI at all.

There is hope, however.  The basic solution looks something like Peter van Eerten’s GTK-Server.

Most every scheme implementation makes some provision for starting external programs and talking to them, or at the very least communicating over TCP.  Once we realize this, our problem becomes vastly simpler.  Our task is no longer to write a standard foreign function interface that unifies all the different schemes out there.  All we have to do is come up with an interface that can start an external program, and communicate with it.  The semantics for this task are much simpler and often much more similar across implementations than those of an FFI.

Of course, just having this library is no good.  If we have that, we’re very well placed to use GTK-Server, but essentially no better off at interfacing with any C library that comes along. To make this work, we’re going to need something to talk to.

That’s what SoLoad is.  SoLoad implements a simple text-based protocol which allows you to do some fancy things with C libraries.  Using SoLoad, you can open libraries, load functions, and call them, as well as messing around with the types and values of pointers.  There is also support for “definition” files, which allow you to load all the defined functionality from a library with a single call.

As well as bringing all of this functionality much closer to portable availability in scheme, the existence of this program has the rather pleasant side effect of making all that functionality availably to any language that has basic process starting and text sending abilities.  Feel like programming the next killer FPS in awk, anyone?

Stay tuned for Part 2, in which we learn how to actually make use of SoLoad.