Making a small Lisp project with quickproject and Quicklisp

A few years ago I wrote about how I make small Common Lisp projects. With the availability of Quicklisp and ASDF2, my process has changed quite a bit. Here's what I do lately.

Get a comfortable environment (done only once, not once per project). First, as part of setting up my CL environment, I install Quicklisp and add it to my SBCL startup file. That means downloading quicklisp.lisp and then running a couple commands:

(load "quicklisp.lisp")
(quicklisp-quickstart:install)
(ql:add-to-init-file)

After that, Quicklisp loads automatically every time I start Lisp, and I have more than 300 libraries available to add as easy dependencies of my project, if needed.

Since I use Emacs and really like slime, I also usually install quicklisp-slime-helper with (ql:quickload "quicklisp-slime-helper") as one of my first steps on a new system. Emacs isn't required to follow the project-creation steps below, though.

Second, Quicklisp includes ASDF2. I like to set up ASDF2 to scan a particular directory tree, ~/src/lisp/, for local systems. To do that, I create a config file named ~/.config/common-lisp/source-registry.conf.d/projects.conf that has this in it:

(:tree (:home "src/lisp/"))

With that file in place, I can add new projects to that directory tree, and after an (asdf:initialize-source-registry) the project's systems will be loadable with asdf:load-system. I can unpack tarballs, check things out from source control systems, or create new projects and they'll all be easily available for loading.

ASDF2's default setup also scans a directory called ~/.local/share/common-lisp/source/, so if you don't mind putting projects there, you can use that without any additional configuration.

With Quicklisp installed and ASDF2 configured, here are the steps I follow when I get an idea and I want to explore it in Common Lisp.

note In the following examples, I use pathnames like #p"~/foo/" to mean (merge-pathnames "foo/" (user-homedir-pathname)). Most CL implementations allow this shorthand, but if yours doesn't, you will need to use the full pathname, e.g. #p"/home/xach/foo/".

Create a project skeleton with quickproject and load it. Quickproject is part of Quicklisp, so if it's not already loaded, I can just use this:

(ql:quickload "quickproject")

For this example, I'll make a project called swatchblade that generates rounded-rectangle PNGs of a particular color, and makes it available as a web service with Hunchentoot. To create a project skeleton for the project, I use this:

* (quickproject:make-project "~/src/lisp/swatchblade/"
                             :depends-on '(vecto hunchentoot))
"swatchblade"

The last part of the directory name is used as the new project name. I could choose a different name by passing the :name option explicitly.

quickproject:make-project creates several files in the swatchblade directory:

  • package.lisp
  • swatchblade.lisp
  • swatchblade.asd
  • README.txt

It also adds the directory to your ASDF configuration, so you can immediately load the skeleton project and its dependencies:

(ql:quickload "swatchblade")

ql:quickload will automatically install required libraries if they're available in Quicklisp.

Write some code. I open the newly-created ~/src/lisp/swatchblade/swatchblade.lisp and start hacking.

I define variables with defvar and defparameter, functions with defun and defgeneric, macros with defmacro, classes with defclass, etc. As I write each one, I compile it immediately with C-c C-c and occasionally switch over to the REPL to run some code.

As I use symbols from other projects, I update the defpackage form in package.lisp to import symbols. For example, I might want to use several Vecto symbols without package prefixes, so I could do this:

(defpackage #:swatchblade
  (:use #:cl)
  (:shadowing-import-from #:vecto
                          #:with-canvas
                          #:rounded-rectangle
                          #:set-rgb-fill
                          #:save-png-stream))

I don't put any library management code directly into Lisp source files. If I decide to use more external projects, I edit swatchblade.asd and add to the :depends-on list, e.g.:

(asdf:defsystem #:swatchblade
  :serial t
  :depends-on (#:vecto
               #:hunchentoot
               #:cl-colors)
  :components ((:file "package")
               (:file "swatchblade")))

Reloading the system with ql:quickload will install (if necessary) and load any newly-required systems.

Reorganize. For small projects, sometimes a single file suffices. Most of the time, though, I end up splitting code up into multiple files. In this example, I might make a file called utils.lisp, a file called graphics.lisp, and a file called web.lisp, and update the system definition from this, the system automatically created by quickproject:

(asdf:defsystem #:swatchblade
  :serial t
  :depends-on (#:vecto
               #:hunchentoot)
  :components ((:file "package")
               (:file "swatchblade")))
...to be something like this:
(asdf:defsystem #:swatchblade
  :serial t
  :depends-on (#:vecto
               #:hunchentoot)
  :components ((:file "package")
               (:file "utils")
               (:file "graphics")
               (:file "web")
               (:file "swatchblade")))

When the :serial t option is present in the defsystem, files are compiled and loaded in order. You can get more complicated in expressing inter-file relationships, but I haven't found it worth the trouble. I just organize my files so that functions and macros needed in later files are provided in earlier files.

Reuse. With something like swatchblade, I would probably re-use it by starting Lisp, loading the project with ql:quickload, and running a function to start Hunchentoot with the swatchbade handler in effect. The final package definition might look something like this:

(defpackage #:swatchblade
  (:use #:cl)
  (:export #:start-web-server)
  (:shadowing-import-from #:vecto
                          #:with-canvas
                          #:rounded-rectangle
                          #:set-rgb-fill
                          #:save-png-stream))

The session then might look something like this:

* (ql:quickload "swatchblade")
loading output
* (swatchblade:start-web-server :port 8080)
Server started on port 8080.

With a project that is meant to be used more as a library, the package would likely have many more exports, and I would re-use it by passing it with the :depends-on argument of quickproject:make-project, e.g.:

(quickproject:make-project "~/src/lisp/whimsytron/" 
                           :depends-on '(swatchblade))

From there I can go back to the "Write some code" step and continue the cycle.

If I want to reuse a project as a standalone program I can run from the command-line, I use Buildapp.

Comments

http://xach.livejournal.com/data/atom should work for subscribing.

One package for each file?

Yes it works.

i have one question in regards to the "Reorganize." section of your post:

I have split the file into two and extended :components keyword paramater in asd file accordingy. Do i have to define a separate package for each file in package.lisp and add the (in-package ...) expression in each file?

Because when i tried to just use the same (in-package ...) as in the first file without defining any additional package i am getting an error when loading the system via quickload, telling me that the package of the second file name does not exist...

In my case it would be enough two have one package for both files.

Thank You
matus

September 2014

S M T W T F S
 123456
78910111213
14151617181920
21222324252627
282930    
Powered by LiveJournal.com