Archive

Posts Tagged ‘repl’

On Clojure, Swig, and the UnsatisfiedLinkError

September 16th, 2011 3 comments

This week I did some Clojure work with a swigged-up C++ library that works great from Java. I was surprised when things didn’t go as smoothly in Clojure. In particular, loading native libraries in the REPL doesn’t seem to work. I’m writing this down in case someone else has the same problem in hopes that it’ll save them an hour or two of head-scratching and library path fiddling.

So, to set the stage, let’s make a tiny Swig library, compile it and try using it from the REPL. I’m using Leiningen just to save some time:

    $ lein new swiggy
    $ cd swiggy
    $ cat >swiggy.i <<EOF
    > %module Swiggy
    > %{
    >   int get_something() { return 42; }
    > %}
    > int get_something();
    > EOF
    $ swig -java -package swiggy -outdir src/swiggy swiggy.i
    $ gcc -I$JAVA_HOME/include -L$JAVA_HOME/lib -shared swiggy_wrap.c -o libswiggy.dylib
    $ javac src/swiggy/Swiggy*.java
    $ export JAVA_OPTS="-Djava.library.path=. -verbose:jni"
    $ lein repl
    ... snipped tons of irrelevant -verbose:jni output ...
    user=> (System/loadLibrary "swiggy")
    nil
    user=> (swiggy.Swiggy/get_something)
    java.lang.UnsatisfiedLinkError: swiggy.SwiggyJNI.get_something()I (NO_SOURCE_FILE:0)

That’s interesting. libswiggy.dylib loaded happily and yet there’s an UnsatisfiedLinkError. The thing to note here is that it’s complaining about linking the get_something() method, not the library itself. That’s unexpected. Also note that I have the -verbose:jni flag set, but nothing was printed about linking the method to the backing native code in the library. hmmm.

So, I googled. And found a couple references to this problem without real solutions. Unfortunately, the usual reaction to this problem is immediately “Did you set LD_LIBRARY_PATH and java.library.path?” The exception is the same and that’s normally the problem in Java. It turns out that putting the loadLibrary() call within a Java class that’s loaded by Clojure is enough to trick the system into correctly linking the native method. That is, edit Swiggy.java something like this:

    ...
    public class Swiggy {
      // Explicitly load the library when this class loads
      static {
        System.loadLibrary("swiggy");
      }
      public static int get_something() {
        return SwiggyJNI.get_something();
      }
    }
    ...

then recompile and give it another try:

    $ javac src/swiggy/Swiggy*.java
    $ lein repl
    user=> (swiggy.Swiggy/get_something)
    [Dynamic-linking native method swiggy.SwiggyJNI.get_something ... JNI]
    42

That’s better. Does anyone know what causes this? Is it related to the classloader used by the repl or something?

If your in the situation where you can’t or don’t want to modify the swig interface file to include this change, all is not lost. Simply create your own dummy Java class with the loadLibrary call and reference it from Clojure. You can even use Leiningen’s :java-source-path property to get it to auto-compile it for you. I suppose gen-class should also work, but the Java’s pretty trivial.

Categories: clojure Tags: , , ,

Compojure, the Repl, and Vars

November 5th, 2010 1 comment

For a future post, I’ve been working on a simple webapp using Compojure, a simple web framework for Clojure. Working with Compojure interactively in the repl gets mentioned a lot, but never with very satisfying answers. So tonight I worked through it and believe I have a reasonable approach that covers everything I want, namely:

  • The ability to start the server in the background of the repl
  • The ability to stop the server in the repl
  • The ability to reload code, hit refresh on the browser, and see my changes

I’ve seen several posts that have a separate namespace (i.e., .clj file) for running the server. This didn’t seem necessary in my tests. Assmuing we’re in web-app.core (src/web_app/code.clj), define your routes and a run function like this:


(defroutes all-routes
  ... route definitions ...)

(defn run
  [options]
  (let [options (merge {:port 8080 :join? true } options)]
    (run-jetty (var all-routes) options)))


There are a couple semi-interesting things going on here. First, we have some default options for the server. When (run) is called, its default behavior is to run the server on port 8080 and block (because of the join? flag, which is important once we’re in the repl).

Secondly, rather than passing all-routes directly to run-jetty, we pass in (var all-routes) (or the #'all-routes reader macro). This way, when we reload our code, the server is using the Var rather than having its own copy. When the root binding changes, the server picks it up without having to restart (see below for a simple example).

Finally, we can use this in the repl:


  ;;; Load the web-app
  user=> (use 'web-app.core :reload)
  nil

  ;;; Run the server with join? false so it doesn't block the repl
  ;;; Also save the result so we can stop the server later
  user=> (def server (run {:join? false}))
  #'user/server
  2010-11-05 20:46:32.797::INFO:  Logging to STDERR via org.mortbay.log.StdErrLog
  2010-11-05 20:46:32.800::INFO:  jetty-6.1.14
  2010-11-05 20:46:32.903::INFO:  Started SocketConnector@0.0.0.0:8080

  ;;; Now test the webapp, make some changes to the code, and ...
  user=> (use 'web-app.core :reload)
  nil

  ;;; Now make sure the changes took effect. Yay!
  user=> (use 'web-app.core :reload)
  nil

  ;;; If we need to, we can stop the server and restart it too!
  user=> (.stop server)
  nil
  user=> (def server (run {:join? false}))
  #'user/server
  2010-11-05 20:48:02.022::INFO:  jetty-6.1.14
  2010-11-05 20:48:02.046::INFO:  Started SocketConnector@0.0.0.0:8080
  nil
  user=> 


That’s about it. Way better than restarting the repl on every code change. I bet there’s much slicker way to do this in Compojure, but I haven’t found it yet.

Var?

As mentioned above, passing the var of the request handler is key to having changes take effect when the code is reloaded. Here’s a very simple example of this behavior outside of the webapp context:


  user=> (defn greet [] (println "Hi!"))
  #'user/greet
  user=> (greet)
  Hi!
  nil
  user=> (defn call [f] (f))
  #'user/call
  user=> (call greet)
  Hi!
  nil
  user=> (def wrap (partial call greet))
  #'user/wrap
  user=> (wrap)
  Hi!
  nil
  user=> (defn greet [] (println "HEY!"))
  #'user/greet
  user=> (wrap)
  Hi!
  nil

  ;;; Note that it still prints "Hi!".
  ;;; Let's try that again, but use var instead

  user=> (def wrap (partial call (var greet)))
  #'user/wrap
  user=> (defn greet [] (println "Hi!"))
  #'user/greet
  user=> (wrap)
  Hi!
  nil
  user=> (defn greet [] (println "Hey!"))
  #'user/greet
  user=> (wrap)
  Hey!
  nil

  ;;; This time the change to greet took affect!


Categories: clojure Tags: , ,