Daniel Weinreb on packages

Daniel Weinreb has posted a number of really interesting things to the "pro" mailing list lately. I particularly like this digression on packages:

(Digression: A slightly different approach is to define a package not intended to be "use"'ed. I did this with the API for the logging facility in QRes. For example: log:add-rule. If you did a "use", it might be confusing to remember that add-rule is from logging, and indeed you might "use" another package that has an "add-rule" function. This is a name conflict. The whole idea of packages is to avoid name conflicts! So I think "use" is opposed to the whole idea of packages and should be used sparingly.

What people usually do in Common Lisp, in my experience, is to name their functions so that the ones in one module (or, more generally, related ones) all have names starting with a common prefix. Well, that's what packages were invented: so that you don't have to do that! In QRes, even though there is a "qconfig" package for the configuration stuff, the functions are still all named "config-", and everybody uses "use". I think this is not as good as using packages as they were designed. Rather than, e.g., config-descriptor-property, use config:descriptor-property. Anyway.)

My experience has been a bit more positive. I think that people use the "bad" style (redundant symbol prefixes for functions, slots, etc) as first instinct when coming to Lisp from other languages, and after a while they use the "good" style.

It would be nice if this transition time could be shortened, not just for naming style but for everything. Practical Common Lisp helps a lot, but it would be nice if there was more promotion of exemplary, modern CL code for people to study.

You can read more of Daniel's recent posts on the "pro" archive.




That's why nicknames are important

What Dan describes is exactly why having nicknames is important. People don't want to fully qualify names when the package name is com.whatever.buzz, and if the package has no nicknames, then it's simpler (or so they think) to just "use" it.

So packages should have short and convenient nicknames. But the problem is that if you do so in defpackage directly, it completely defeats the idea of avoiding name conflicts! This means that a package author should never decide on the nicknames (in fact, the :nicknames option to defpackage is totally evil).

The solution I use in Clon is to provide this function: (nickname-package &optional nickname).
Without argument, the com.dvlsoft.clon package gets "clon" as a nickname and suddenly your lisp code is shorter. But if that would conflict with one of the other Clon libraries for instance, then you can choose an alternate (still short) nickname like "cln".

The documentation advertises this and encourages people to call nickname-package right after asdf-require'ing it.

-- didier verna


using packages, and lisp style

Isn't there a style-guide, somewhere? Maybe there should be a short "coming from another language?" text, with suggestions on how to make the transition to lispy style.


Packages aren't really lispy

I suspect package issues get little attention because they're neither beautiful not particularly distinctive to Lisp. It's fun (and useful) to explain the lispy features - functional style, DSLs, how to write macros, etc. - but the package system is not something we're proud of, so we don't talk about it.

Re: Packages aren't really lispy

This sounds like Paul Graham brain damage to me.


Re: Packages aren't really lispy

The brain damage being the tendency to be more interested in the idealized language than in the dialect you're using? I don't think that's because of Paul Graham; Lispers have always done that, and it's probably a good thing, when it leads people to fix problems rather than tolerate them. But maybe the unpopularity of the package system is a red herring here, because the ugly parts are not the parts that attract usage advice.

The basic package advice (in-package; not adding prefixes to everything; not using up the short nicknames if you're making a library for distribution) applies to most module systems, so it may be overlooked for not being Lisp-specific. Some popular practical advice (defpackage in a separate file; very unique package names; nicknames instead of use) can be ugly and inconvenient and has no immediate benefit, so it's hard to convince people it's the right way. User-defined nicknames are as good a solution to name collisions as any language has, though, so maybe the standard advice should suggest rename-package or Didier's nickname-package.

Of course, the ideal would be something like (ql:quickload "foobarbaz" :nickname :fbb), if Quicklisp were omniscient enough to know which package to rename. :)

- arcanesentiment

Re: Packages aren't really lispy

The brain damage is not knowing much about Common Lisp as people normally write it, and thinking Platonic Lisp advice is useful or interesting.


nicknames, even when chosen by the user affects the global environment.

Having the user pick the nickname is still problematic as the user might be a library writer. If I user her library, I'm stuck with the nickname as well. Really what's needed are :use-as names that can be used with defpackage instead of :use.

Something more that is more of a hack, but possibly usable with the current package system would be some kind reader-macro in combination with named readtables to do the aliasing. If I use what is effectively a private, but mostly standard readtable for my code, then I can augment the readtime environment in isolation.


Re: nicknames, even when chosen by the user affects the global environment.

I agree. Using a private readtable is something I had planned to do in case the problem occurs someday (clon is already prepared for this: every called to (in-package :com.dvlsoft.clon) is immediately followed by a call to (in-readtable :com.dvlsoft.clon).

Re: nicknames, even when chosen by the user affects the global environment.

IMO, Ron Garrett/Erann Gat's lexically scoped packages get this more right.

Unfortunately, it's hard to see how we'd get to a point where people actually started using these....


Best practices

i think it would be great to have a collection of "CL best practices" such as this one.

XML namespaces, C++ namespaces, Java namespaces

It's interesting to compare and contrast the different approaches.

In XML, the short names are mandatory -- but purely specified at point-of-use. Not just by the user, but scoped to the scope of the usage, so you can compose XML documents from different sources without conflicts for the short names.

C++ takes a different approach. It doesn't have short OR long names, but has a hierarchy, again with scoping, and use -- which can be applied either for an entire namespace or an individual name within.

In Java, the the implicit scoping is a bit more complex. You can take Java packages as being perhaps the analog of C++ namespaces -- with directory scope rather than an explicit construct that can appear anywhere. Java also has similar capabilities for using symbols from other namespaces.

Common Lisp has the worst of all worlds. Our only defense is, well, we were the first to really tackle the problem! Short names are assigned by the definer of the source package, rather than the consuming package. They're global, as are packages themselves.

There's no scoping whatsoever, except for that implied by in-package in various places in your files -- and tools aren't going to respond well to having more than one.

Part of the difficulty is the intermixture of processing and reading involve. in-package is a compile-/eval-time amalgam that affects subsequent *parser* behavior.

I would claim it really ought to have been done via a *reader directive*. Maybe if always wrote #.(in-package "foo"), we'd be better off -- but anything that reads s-expressions from a file would need to bind *package* (as LOAD already does). Better would be a syntax that states the scope.

Perhaps something like:

#{ {"MyPackage" (:using "YourPackage" "u")}

This would, in the reading of (expr1) and (expr2), use MyPackage as the current package, and also make "YourPackage" available with the "u:" prefix. This would not be a modification of MyPackage! Rather, you'd want a new variable to hold a current hashtable of local prefixes, perhaps *local-package-prefixes*.

Then a tool which reads a source file would not need to concern itself with packages.

Tools that want the package usage information would need to register callbacks to be informed by the reader -- something else that's missing from the language. Many tools would like to be notified of the correspondence between source and parse tree, to bridge the textual and semantic representations.

Re: XML namespaces, C++ namespaces, Java namespaces

I finally figure out how to use OpenID to link it to my Typepad account, only to find that it doesn't give the reader a clue who posted. FAIL!

-- Bob Kerns

July 2014

Powered by LiveJournal.com