Logical pathnames survey

How would you describe your relationship with Common Lisp's logical pathname facility?

  1. I use logical pathnames and they make my life easier. Here's how...
  2. I used logical pathnames and they made my life miserable. Here's how...
  3. I never tried logical pathnames. Here's why...

Please comment on my blog with your response, or send me an email, and I'll summarize the responses later. (If you don't use logical pathnames because you don't use Common Lisp, no need to respond.)

(Personally I'm in camp 3, because pathnames seem complicated and logical pathnames were lumped in with them in my mind. And I didn't really understand what problem they were solving. But now that I've read about them and how they can be used, I think I'll start using them and be in camp 1.)




1 and 2

I used logical-pathnames on ACL, where they seem to coexist happily with modern file systems (ACL quietly violates the spec and uses case-sensitive pathname components).

Then I ported code from ACL to SBCL, and moved from Camp #1 to Camp #2.

I bet if you use them, you will find yourself in camp #2, especially if you're primarily an SBCL user.

Alas, ASDF doesn't really happily provide a data file solution (uses of *load-truename* in your source files blows up spectacularly when people use the binary-relocation features of ASDF, which are essential to anyone using more than one lisp implementation). An obvious extension opportunity.



Re: 1 and 2

Actually, ASDF _does_ let you find "data" files:

(asdf:defsystem #:example
  ((:static-file "data")

Then, you can find the data's location with a quick little:

(asdf:component-pathname (find "data" (asdf:module-components (asdf:find-system '#:example))
                               :test #'string= :key #'asdf:component-name)))

I hope there's a more direct way to do this. Maybe it'd be nice if ASDF set up your logical pathname translations for you already, so you could specify a translation in the system definition, and then just load #P"example:data".

Speaking of logical pathnames -- I'm in group 3, although Clozure CL, which I use regularly, uses a logical pathname translation for the #P"ccl:" host that points to its own source.


Re: 1 and 2

It'd be really handy if asdf did that, and you could rely on it. Though I don't think it could snarf the entire host space like that, it'd probably have to be e.g. asdf:example;data


Re: 1 and 2

Well, ASDF does include a find-component generic function as a small improvement:

(asdf:component-pathname (asdf:find-component (asdf:find-system '#:example) "data") )

Also, if your data file isn't identified explicitly in your system definition, then you can still reference it using asdf:system-relative-pathname:

(asdf:system-relative-pathname (asdf:find-system '#:example) "path/to/foo.dat" )

Of course, neither of those pathnames are particularly logical, are they? ;)
camp 3, never had a situations when need use logical pathnames.
I never used them because I don't write Lisp that much and I don't know what they are. Unless they're the thing that's outlined in Practical Common Lisp in which case I don't use them because I don't write stuff that needs to go on a system other than mine.


Camp 3

I'm in camp 3, for reasons very similar to the ones you outlined. Also, the problem it appears they're intended to solve--aiding portability of systems from one implementation to another--has really never come up for me.

Logical pathnames are one of the corners of CL I've always meant to really explore, but to be honest, I've never more than glanced at it. (Which is to say that I'm still interested, and one day... ;-)


Camp 3

I've not written anything that makes extensive use of the filesystem where I suspect that the added complexity is worth it.

I wonder though what it is you've read that may put you in camp 1...
Files in general have always been a source of irritation for me, and it was never clear to me how logical pathnames are mapped to actual pathnames. I always assumed it would be yet another configuration item to have to deal with (like asdf's central registry variable).


Door number three

Camp 3: never used them.

They sound useful in the abstract, but in practice pathnames are one of the more portable aspects of modern systems. Every system uses either a Unix filesystem, or something pretty darned close to it -- I've never actually used a filesystem with version numbers, for example. The examples in CLTL2 look totally unlike anything I've ever needed to do.


I'm in camp 1. We use logical pathnames for a system that originally ran on MCL and was ported to CCL/lispworks. By using logical namestrings consistently, we only need to condotionalize the definitions of logical hosts, rather than all uses of namestrings.



I use logical pathname successfully, by restricting myself to strict conformance. This means that I bear with the implementation idiosyncrasies, and don't try to access random files in the underlying file system (but thru explicit logical pathname translations).
I am in camps 1 *and* 2.

I use pathnames because they are the best "abstract filename abstraction" out there. Unfortunately, they try to abstract all file names, everywhere, with some features available in some (but far from all) operating systems (like file revisions, something that is natively available in VMS and, I believe, DEC-system 10 and 20, but certainly not natively available in unix even if you can, to some extent, mimic this with naming conventions).

So, yes, there are problems and I have vivid memories of swearing at the pathnam system. However, when you're processing and changing filenames, it's much easier swearing a bit than it is to write icky string-parsing code.
In camp 3: I don't program in Common Lisp so much to use such features.

Firmly in camp 1, given some rules

I've successfully used logical pathnames to replace a shell script that had to rely /heavily/ on basename/dirname games in order to construct semi-correct directory names. Using LPs, I can easily do things such as extract a namestring relative to some base directory (using enough-namestring), and construct pathnames from existing ones that feature components that differ slightly in the middle.

There are rules that I learned to follow, however, to minimize pain with at least SBCL.

* Don't use the logical pathname host, don't use LP name strings.

* Don't use the :version slot, leave at :unspecific.

* Really don't use the LP host: The rules for LP-hosted pathname handling specified in ansi cl make it just not worth the pain.

Re: Firmly in camp 1, given some rules

What pain comes from using the logical pathname host? I've read the rules and I can't figure out where the pain would come from.


Camp 2/3

I tried to use them, but found them rather baroque, and they seemed to need too much effort to learn compared to the problem they solved. Since I did not need to know the details often, I kept forgetting the subtleties, finding that I was needlessly wasting time when I tried to use them. Using strings, in contrast, always gets the jobs I need to do done in almost no time.



Logical pathnames survey

In Camp #2, with movements into #1. Coming from a Unix background the
complexity is baffling, I can easily and concisely describe a pathname
already. Right? Then I found myself needing to use and distinguish
between relative paths, absolute paths and canonical paths in a tree
of symlinks (with the wrinkle that they differ between environments).
It's a pain but logical paths make it slightly easier to handle and
I don't have to write code to manipulate the path atoms as strings.

- Ross


Camp 1/2

Used to be in Camp 2, because it took me a bit of head-scratching until I realized the limits of LPNs, and that it's futile to use them outside these limits, e.g., filenames with multiple dots, mixed-case filenames, etc.

Camp 1, because some things are nice:

  • (ed "sys:src;") in the SLIME repl brings up a dired buffer to the sbcl sources. I've set up LPNs to make (mapc 'ed '("home:" "lisp:" "home:code;")) work, too, regardless whether I'm on Linux or OS X.

  • You can have some fun with fasls:
    CL-USER> (compile-file "home:code;test")
    ; compiling file "/Users/michaelw/Documents/Code/Lisp/test.lisp" (written 24 NOV 2004 10:48:26 PM):
    ; compilation finished in 0:00:00.035
    CL-USER> (load "home:code;test" :verbose t :print t)

    It would be even better if COMPILE-FILE would create non-existent directories (in sbcl, it does not)...

Besides that, I am not using LPNs much, because my needs are simple.

I haven't bothered with a logical pathname since I left Harlequin and was no longer being paid to care about them. I've never understood what the benefit was.


I'm in camp #3, mainly because there's enough fear and loathing in me when it comes to pathnames that I haven't really looked at logical pathnames seriously. I have a vague understanding of their function, but that's pretty much it. I might try them if I ever have a piece of software with significant requirements for path-related configuration, but even then I'm not stepping anywhere near without a long and deep look at gotchas first.


Jānis Džeriņš; guess I'm in the camp #1

I use DIRECTORY with wild (and very wild, as in :wild-inferiors) logical pathnames. For instance, given following function definition:

(defun list-files (ext &optional (root *some-default-root*))
  (merge-pathnames (make-pathname :directory '(:relative :wild-inferiors)
                                  :name :wild
                                  :type ext)

I can get all the text files (recursively!) in my home directory like this:

(directory (list-files "txt" (user-homedir-pathname)))

This method suits me fine because in my case I'm working with all the files. But this not the best way if not all the file names are needed (especially if the directory tree is big).

Camp 3

I haven't bothered with a logical pathnames. Always worked with unix-like environments and never understood/had time to learn the benefits of pathnames.


Camp 3

I've been hacking in Lisp for 3 years or so and I have never tried using logical pathnames. I have never even read about them. Although I'm all ears if someone can explain their usefulness to me.


Was in camp 2, now 3

LPNs caused me grief until I started paying more attention to the spec and realised that (notwithstanding that some implementations provide considerably more than the spec says they have to) they don't do what I thought they would do anyway. Now I don't use them, because the things I thought they should do are the things I wanted them to do and I don't ned to do the things they do do

I would use LPNs under these conditions

* When all of the files will be created by the same Lisp implementation and only ever accessed using that Lisp implementation
* and I can name them all using only uppercase letters, digits and the hyphen (-)
* and I don't care too much about how they're represented in the underlying file system

and never for accessing files that were not created by the same Lisp implementation. Example

* (translate-logical-pathname #p"cl-library:src;SomeJavaClass.java")

Yes, this is an attitude born of using SBCL. If it didn't fold case (as the spec requires) they would probably be a whole lot more useful in practice - as indeed they are on other implementations that don't follow spec on this point.


#3; don't see what they buy me

I like the basic idea of logical pathnames; define an abstraction so the CL runtime can map a name to something approporiate for the particular implementation and machine. I don't like the specification; it overly restricts that mapping. I think it was specified at the wrong level.

I'd love to be wrong in my interpretation; but previous comments about the do's and dont's seem to confirm my understanding. Most things I want are better(?) layered on top of *default-pathname-defaults* and the list form for specifying directories.

IMO, CL would benefit from two filesystem improvements.
1) a well-specified mapping from physical pathnames to each filesystem in use (i.e. a POSIX mapping, a fat/ntfs mapping, etc.)
2) a more abstract interpretation of logical pathnames, with portable hooks that can interpret paths in arbitrary ways to specify files, urls, the contents of a version-control system, the contents of an archive, a lazy function stream, etc. The only requirement is to behave sanely for open, read, write, etc. An early rewrite filter would provide support for the current logical pathname framework and could be implemented by a portable library.

The first might be possible through a portable library layer that translates to each implementation's behavior, but I think it really belongs in the implementation (possibly with a backwards-compatibility flag). The second seems to need implementation support.


Use them occasionally. Love them. People who have problems with them (including thinking they're restrictive -- they're not; you just have to write the translations properly) just don't understand them.

July 2014

Powered by LiveJournal.com