Zach Beane's Blog

Cody and Clint Phipps, aka Pitch Black Ribbons.

The code in Quicklisp represents a large amount of interesting data. It's difficult to systematically query it, but I've been trying to make it easier, and one step in that process is available as a library called qlmapper.

qlmapper is pretty simple; it can load an arbitrary Lisp file after loading each system in Quicklisp. Each system is loaded in a fresh SBCL instance, so code that inspects and reports things can work from a mostly-clean Lisp environment.

Here are some pieces of information you could gather and share:

• What packages does a given system define?
• What are all the packages defined in Quicklisp? What systems introduce conflicting package names?
• What foreign libraries does a system load? What is the name of the Debian package name that provides that foreign library?
• Everything that Manifest does
• Does any code use nreconc or revappend? (Or, more generally, a CL-aware code search engine.)
• What ASDF system definitions include :author/:description/:license metadata? Which system definitions need to add it?
• Which projects lack a README file (or some variation thereof)?
• Which projects don't build and why?

Here's a small example script that I just made for qlmapper:

(defpackage #:foreign-report
  (:use #:cl))

(in-package #:foreign-report)

(defun canonical-name (library-pathname)
  (let* ((name (file-namestring library-pathname))
         (end (search ".so" name)))
    (subseq name 0 end)))

(defun find-library (line)
  (when (and (search "r-xp" line)
             (search ".so" line))
    (let ((path-start (position #\/ line)))
      (when path-start
        (subseq line path-start)))))

(defun maps-table ()
  (let ((table (make-hash-table :test 'equal)))
    (with-open-file (stream "/proc/self/maps")
      (loop for line = (read-line stream nil)
            while line do
            (let ((library (find-library line)))
              (when library
                (setf (gethash (canonical-name library) table) library)))))
    table))

(defun foreign-mappings ()
  (let ((table (maps-table)))
    (loop for object in sb-sys:*shared-objects*
          for name = (sb-alien::shared-object-namestring object)
          collect (list name (gethash (canonical-name name) table)))))

#+qlmapper                                                                      
(with-open-file (stream "~/foreign-libraries.sexp"                                  
                        :direction :output                                      
                        :if-exists :append                                      
                        :if-does-not-exist :create)                             
  (let ((mappings (foreign-mappings)))                                          
    (when mappings                                                              
      (print (list cl-user:*qlmapper-object-name*                               
                   mappings)                                                    
             stream))))

I called it with this: (qlmapper:map-loaded-systems "~/foreign-report.lisp")

An hour later, it produced an interesting report of library usage. It's not perfect, but it's a start, and can be refined to provide more accurate and useful answers. Even in this raw form, I can tell some interesting things. For example, I can tell which libraries I had to build from source (no Debian package available) by checking for "/usr/local" in the results.

What other stuff would be fun to discover about the universe of Quicklisp code? What changes and improvements to qlmapper would make it even easier to discover?

I just pushed qlmapper to github.

It's made for calling some Lisp code for each system in Quicklisp in a completely fresh SBCL environment. I'm going to use it to gather descriptions and documentation for each project. I hope other people find it useful to systematically work on Quicklisp projects in some way.

Enjoy!

Are you in Los Angeles? Do you want to hang out with Lisp nerds? CRACL is meeting on April 19th after a lengthy hiatus.

Mathematical Systems, Inc is a Japanese company that has been doing CL stuff for a long time. They've just recently started posting some of their CL libraries to github.

(Of course, you already knew that if you read the github CL project feed.)

For a long time, Allegro CL has allowed a syntax like this:

foo::(bar baz quux)

The "foo::" prefix reads the subsequent sexp as though the current package was the "foo" package.

SBCL 1.0.55 introduced support for this syntax as well. I've started to use it more and more, because I sometimes enter stuff like this at the REPL:

(ql-dist:ensure-installed (ql-dist:system "foo"))

Now I can use this:

ql-dist::(ensure-installed (system "foo"))

It saves me the trouble of inheriting or importing symbols, or switching packages, for simple one-off stuff in the REPL. I don't use it in source files — I don't feel like being that gratuitiously unportable.

Thanks to Nikodemus Siivola for adding this capability to SBCL.

This is 11 days ago. Yesterday the ice cream man made the rounds in the (completely snow-free) neighborhood. Welcome to Maine!