Package-inferred systems follow a useful one-file-per-package convention structure. But package-inferred systems themselves are harmful and should not be used. assets/package-inferred.png Image or a box, drawn by an obvious amateur without knowledge of perspective. Inside the box, there’s a stereotypical time bomb with a clock face and three TNT sticks. Near the box, there’s a card with “aartaka.me” on it, and a letter “From: Artyom Bologov”. I swear I’m not a terrorist, I’m just a blogger! IMAGE_ALT

Package-inferred systems are an ASDF (Common Lisp build system) extension. This extension results from a popular style of Common Lisp programming: spawning one package per file and controlling its imports per file/package. While I don’t necessarily like this style, I have to acknowledge its benefits:

Yet this style of programming might also result in a slew of bad practices. Including difficulty of packaging and undisciplined package pollution. Making package-inferred systems more dangerous than plain systems. Be wary of these.

Discipline and Wrong :use

The way ASDF manual describes package-inferred systems extension is: Each file under the my-lib hierarchy will start with a package definition. The form uiop:define-package is supported as well as defpackage. ASDF will compute dependencies from the :use, :mix, and other importation clauses of this package definition. 6.5 The package-inferred-system extension>

Package-inferred systems are pretty useful as they are, parsing packages in a DWIM way. But they only handle defpackage and uiop:define-package. Not custom package definition constructs like nyxt:define-package. Incomplete by design.

This description is already problematic: it suggests :use and :mix clauses. Both of these are overly expansive, importing all symbols from listed packages into the current one. In case of :mix, conflicting symbols are also getting shadowed on conflict.

:use (and, by extension, :mix) is a bad practice, because it obscures symbol sources. And bloats packages with unused/unnecessary symbols. In case one needs easy access to package’s symbols, one should use package-local nicknames. These are available on all currently maintained CL implementations. And they still highlight that the symbol comes from elsewhere.

Not using :use is a matter of discipline. But, when even the ASDF manual highlights this as the the solution, and when this is the easiest way to import symbols without worrying about granularity… One will use :use (sorry for the tautology.) No use (I really need to stop) in preaching discipline.

IDE-less Reading

It might be a misplaced life balance on my side. But I often spend time idly reading through library sources. Looking at their algorithmic choices and style. Maybe there’s even something I can learn: CL is a big language and I sure don’t know all of it!

I don’t even load the library into the image sometimes. It might not be comprehensible to me, or the domain would be too alien. So IDE-less (or, rather, Sly-less) reading it is.

(Just so you understand the depth of my IDE-less madness: This post is edited in ed(1), the standard text editor. I’m having an Emacs abstinence month, and I am kinda fine with ed(1) anyway.)

One problem with this IDE-less reading is that symbols don’t have source locations. This is fine, because I can often grep the sources for the necessary defun or defmethod. No biggie. But with package-inferred systems, I’m no longer sure. Does this symbol belong to current file / package / system? Does it come from a dependency? I don’t know all Alexandria symbols, and I sure am clueless about Lparallel names!

Package-inferred systems promote a style that requires a full-blown Lisp IDE. One can no longer perceive the library or app statically. One has to load / build it. And that puts a huge burden on them, and innocent bystander just wanting to check the style gotchas.

Packaging Hell

This partially stems from the need to load the system: packaging package-inferred systems (catch the irony here?) is hell. I’ve been there, I tried to package Lem for Guix. Unless one loads the system and interactively queries it for dependencies, they risk missing some dependency when packaging. And, as a library packager, one is not always a Lisp programmer or, let alone, Lisp professional.

They just want to package the software. And package-inferred systems make it hard. Load the system or scour the sources for all the :use clauses and deps. Deps, some of which might be unavailable or outdated on Quicklisp. With packages, that don’t always match system names. Needing extra build steps or (god help us) Roswell setup.

Package-inferred systems make packaging harder, because they hide and dilute dependency information. One cannot easily get a full list of deps unless they are proficient in Lisp. Not all packagers are.

Let packagers do their work.

Systems are Metadata, not Data

ASDF is a build system for Lisp, made in Lisp. No, really, it’s not just written in Lisp, it actually is just a library you load to load other libraries. In the same REPL. I sometimes forget what a miracle Lisp world is.

But this easy blend of systems and packages into one symbolic soup is misleading. Systems are metadata of the library or application as a packaging/build artifact. Concerning themselves with files and build sequences. Packages are symbol bags. Involving code-level resource sharing, reference, and visibility. These are two different worlds. Kinda like a work of DevOps is different from a programmer’s one.

One case of system clearly not matching the package is trivial-features. It doesn’t have a package of its own! It exports no symbols! It doesn’t provide any API! Using it in package-inferred model is simply meaningless, because there’s nothing to use.

Package-inferred systems break an already thin boundary between systems and packages. Making these ontologically different things tangled up.

Avoid Package-inferred Systems

I hope this list gives you at least a doubt in package-inferred systems’ usefulness. Note that I’m not against the one-package-per-file style. I’m against lack of discipline and transparency in dependency management. Let’s be kind to each other and not push opaque and dependency-diluted systems on people.