I'm working on FFI for TXR Lisp. Until this morning, it was in the stage of "collection of working API functions, written in C, exposed with Lisp bindings".

I have almost everything I want working, including callbacks. There is a decently rich type system which supports pointers that annotate data passing directions for automatic malloc and free, and this works in both directions: out calls and callbacks (closures).

I support by-value passing and returning of structures. And also of C arrays! Even though C doesn't have by-value arrays. You see, my FFI type (array 42 int) is actually equivalent to the C type struct __anon { int __anon[42] } and not the C type int [42].

Just this morning, between 6 and 6:45 a.m I wrote a partial implementation of the FFI declaration language. Here is what it looks like, with actual output:

Load the under-construction FFI macros:

  (load "ffi")

Define a show macro for showing an an expression and its value as expr -> value:

  (defmacro show (expr)
    (let ((val (gensym)))
      ^(let ((,val ,expr))
         (format t "~s -> ~!~s\n" ',expr ,val)
         ,val)))
  

Now the FFI test. Some structures we need, and declarations of FFI typedefs for them giving type info. The above typedefs aren't required; we we could just write out the (struct ...) syntax inline in the later FFI declarations.

  ;; Lisp struct representing C struct ldiv
  (defstruct ldiv nil quot rem)
  ;; FFI typedef for it
  (deffi-type ldiv (struct ldiv (quot long) (rem long)))
  
  (defstruct pipe nil rfd wfd)
  (deffi-type pipe (struct pipe (rfd int) (wfd int)))
  
  (defstruct dirent nil
    ino off)
  (deffi-type dirent (struct dirent
                             (ino int)
                             (off int)
                             (nil (array 62 int))))
                             ;; the member named nil here is anonymous padding

Now, declare some FFI functions. Very succinct and expressive:

  (with-dyn-lib (libc "libc.so.6")
    (deffi c-abs "abs" int (int))
    (deffi ldiv "ldiv" ldiv (long long))
    (deffi c-getenv "getenv" (buf 3) (str))
    (deffi c-getenv-2 "getenv" str (str))
    (deffi wcslen "wcslen" ulong (wstr))
    ;; Two ways to call pipe: using a pointer to two-int
    ;; structure, or an int[2] array. They are binary identical.
    (deffi c-pipe "pipe" int ((ptr-out pipe)))
    (deffi c-pipe-2 "pipe" int ((ptr-out (array 2 int))))
    (deffi c-read "read" int (int buf int))
    (deffi c-read-str "read" int (int (ptr (array 10 char)) int))
    (deffi opendir "opendir" cptr (str))
    (deffi readdir-r "readdir_r" int (cptr (ptr-out dirent) (ptr-out cptr))))

Finally, the test: with two lines of interactive input from the TTY to satsify two read calls from file descriptor 0. For both of these, I enter the text abc[Enter]:

  (show (c-abs -42))
  (show (ldiv 47 6))
  (show (c-getenv "HOME"))
  (show (c-getenv-2 "HOME"))
  (show (wcslen "foo bar"))
  (let ((pi (new pipe)))
    (show pi)
    (show (c-pipe pi))
    (show pi))
  (let ((pi (vector 2)))
    (show pi)
    (show (c-pipe-2 pi))
    (show pi))
  (let ((buf (make-buf 5 #xa0)))
    (show buf)
    (show (c-read 0 buf (length-buf buf)))
    (show buf))
  (let ((str "xxxxxxxxxxx"))
    (show str)
    (show (c-read-str 0 str 10))
    (show str))
  
  (let* ((de (new dirent))
         (dp (cptr 0))
         (dir (show (opendir "."))))
    (show (readdir-r dir de dp))
    (show de)
    (show dp))

Output:

  (c-abs -42) -> 42
  (ldiv 47 6) -> #S(ldiv quot 7 rem 5)
  (c-getenv "HOME") -> #b'2f686f'
  (c-getenv-2 "HOME") -> "/home/kaz"
  (wcslen "foo bar") -> 7
  pi -> #S(pipe rfd nil wfd nil)
  (c-pipe pi) -> 0
  pi -> #S(pipe rfd 4 wfd 5)
  pi -> #(nil nil)
  (c-pipe-2 pi) -> 0
  pi -> #(6 7)
  buf -> #b'a0a0a0a0a0'
  abc <-------------------------------------- TTY input
  (c-read 0 buf (length-buf buf)) -> 4
  buf -> #b'6162630aa0'
  str -> "xxxxxxxxxxx"
  abc <-------------------------------------- TTY input
  (c-read-str 0 str 10) -> 4
  str -> "abc\nxxxxxx"
  (opendir ".") -> #<cptr: a34e118>
  (readdir-r dir de dp) -> 0
  de -> #S(dirent ino 669944 off 1)
  dp -> #<cptr: a32a258>

Look what is happening in (c-read-str 0 str 10). We pass in a Lisp string which contains "xxxxx...". Magically, it becomes "abc\nxxxxxx..." after the read, as if we were working in C. This is because the type for the parameter (ptr (array 10 char)). The FFI framework recognizes arrays of char and makes them correspond to strings in a two-way manner.