Making a small Lisp project with quickproject and Quicklisp

A few years ago I wrote about how I make small Common Lisp projects. With the availability of Quicklisp and ASDF2, my process has changed quite a bit. Here's what I do lately.

Get a comfortable environment (done only once, not once per project). First, as part of setting up my CL environment, I install Quicklisp and add it to my SBCL startup file. That means downloading quicklisp.lisp and then running a couple commands:

(load "quicklisp.lisp")

After that, Quicklisp loads automatically every time I start Lisp, and I have more than 300 libraries available to add as easy dependencies of my project, if needed.

Since I use Emacs and really like slime, I also usually install quicklisp-slime-helper with (ql:quickload "quicklisp-slime-helper") as one of my first steps on a new system. Emacs isn't required to follow the project-creation steps below, though.

Second, Quicklisp includes ASDF2. I like to set up ASDF2 to scan a particular directory tree, ~/src/lisp/, for local systems. To do that, I create a config file named ~/.config/common-lisp/source-registry.conf.d/projects.conf that has this in it:

(:tree (:home "src/lisp/"))

With that file in place, I can add new projects to that directory tree, and after an (asdf:initialize-source-registry) the project's systems will be loadable with asdf:load-system. I can unpack tarballs, check things out from source control systems, or create new projects and they'll all be easily available for loading.

ASDF2's default setup also scans a directory called ~/.local/share/common-lisp/source/, so if you don't mind putting projects there, you can use that without any additional configuration.

With Quicklisp installed and ASDF2 configured, here are the steps I follow when I get an idea and I want to explore it in Common Lisp.

note In the following examples, I use pathnames like #p"~/foo/" to mean (merge-pathnames "foo/" (user-homedir-pathname)). Most CL implementations allow this shorthand, but if yours doesn't, you will need to use the full pathname, e.g. #p"/home/xach/foo/".

Create a project skeleton with quickproject and load it. Quickproject is part of Quicklisp, so if it's not already loaded, I can just use this:

(ql:quickload "quickproject")

For this example, I'll make a project called swatchblade that generates rounded-rectangle PNGs of a particular color, and makes it available as a web service with Hunchentoot. To create a project skeleton for the project, I use this:

* (quickproject:make-project "~/src/lisp/swatchblade/"
                             :depends-on '(vecto hunchentoot))

The last part of the directory name is used as the new project name. I could choose a different name by passing the :name option explicitly.

quickproject:make-project creates several files in the swatchblade directory:

  • package.lisp
  • swatchblade.lisp
  • swatchblade.asd
  • README.txt

It also adds the directory to your ASDF configuration, so you can immediately load the skeleton project and its dependencies:

(ql:quickload "swatchblade")

ql:quickload will automatically install required libraries if they're available in Quicklisp.

Write some code. I open the newly-created ~/src/lisp/swatchblade/swatchblade.lisp and start hacking.

I define variables with defvar and defparameter, functions with defun and defgeneric, macros with defmacro, classes with defclass, etc. As I write each one, I compile it immediately with C-c C-c and occasionally switch over to the REPL to run some code.

As I use symbols from other projects, I update the defpackage form in package.lisp to import symbols. For example, I might want to use several Vecto symbols without package prefixes, so I could do this:

(defpackage #:swatchblade
  (:use #:cl)
  (:shadowing-import-from #:vecto

I don't put any library management code directly into Lisp source files. If I decide to use more external projects, I edit swatchblade.asd and add to the :depends-on list, e.g.:

(asdf:defsystem #:swatchblade
  :serial t
  :depends-on (#:vecto
  :components ((:file "package")
               (:file "swatchblade")))

Reloading the system with ql:quickload will install (if necessary) and load any newly-required systems.

Reorganize. For small projects, sometimes a single file suffices. Most of the time, though, I end up splitting code up into multiple files. In this example, I might make a file called utils.lisp, a file called graphics.lisp, and a file called web.lisp, and update the system definition from this, the system automatically created by quickproject:

(asdf:defsystem #:swatchblade
  :serial t
  :depends-on (#:vecto
  :components ((:file "package")
               (:file "swatchblade"))) be something like this:
(asdf:defsystem #:swatchblade
  :serial t
  :depends-on (#:vecto
  :components ((:file "package")
               (:file "utils")
               (:file "graphics")
               (:file "web")
               (:file "swatchblade")))

When the :serial t option is present in the defsystem, files are compiled and loaded in order. You can get more complicated in expressing inter-file relationships, but I haven't found it worth the trouble. I just organize my files so that functions and macros needed in later files are provided in earlier files.

Reuse. With something like swatchblade, I would probably re-use it by starting Lisp, loading the project with ql:quickload, and running a function to start Hunchentoot with the swatchbade handler in effect. The final package definition might look something like this:

(defpackage #:swatchblade
  (:use #:cl)
  (:export #:start-web-server)
  (:shadowing-import-from #:vecto

The session then might look something like this:

* (ql:quickload "swatchblade")
loading output
* (swatchblade:start-web-server :port 8080)
Server started on port 8080.

With a project that is meant to be used more as a library, the package would likely have many more exports, and I would re-use it by passing it with the :depends-on argument of quickproject:make-project, e.g.:

(quickproject:make-project "~/src/lisp/whimsytron/" 
                           :depends-on '(swatchblade))

From there I can go back to the "Write some code" step and continue the cycle.

If I want to reuse a project as a standalone program I can run from the command-line, I use Buildapp.


Managing development of libraries

This is a very nice overview of how to start a project.

However, I just wonder how people usually manage the development of their own libraries. Specifically, when you need a "stable" version and a "development" version. For example, I have right now three personal libraries where I need to keep them "frozen" so that I can keep running some experiments for a project I am working on. But at the same time, I need to change/improve some of the code in these libraries for future projects.

Is is just a case of using the vcs (e.g., git), or having different system names (e.g., lib1-stable and lib1-dev), a combination of several things, or something else?


Re: Managing development of libraries

I would suggest using git's branches.

September 2014

Powered by