# 📎 Common Lisp Dependency Vendoring with Submodules By Artyom Bologov (A bright thumbnail with handwritten digital text on it. In the center, there’s a composition of four schematic boxes of differing sizes spread around. Some have lambdas (Greek letter, a symbol of functional programming and Lisp) drawn on them. A bright “SUBMODULE” is written over them. In the corners, attributions to “aartaka.me” and “Artyom Bologov” are written.) So there are half a dozen of Common Lisp package/system/project managers: • Quicklisp (with Ultralisp distribution) • CLPM • OCICL • Qlot • Vend • Qi These mostly fall into two categories: • Package/project managers fetching things from the Internet right into the running Lisp image and not touching the project itself. • Project-local package managers that include all the dependencies into the project tree. Project-local solutions bloat the checkout size and slow down repository cloning. Online ones require network connection. I’m not satisfied with either of the trade-offs, and I think I have a solution to that. Okay, I’ll need a more involved problem statement than that. The features I’m looking for in my dependency management journey are: • Fully local build options • CLI/UNIX-friendliness, especially for running tests • Reproducibility, of a weaker kind—Lisp deps that don’t change underneath me • Not needing bloated Swiss knife binaries just to manage dependencies • Option to use both local deps and Quicklisp / Ultralisp / CLPM / whatever • Custom REPLs with dependencies pre-loaded • Not reinventing the wheel ## Submodules (#submodules) Git submodules are this kind of solution. • They allow the library developer to pin dependencies precisely • The library user only needs to fetch the dependencies they require, when they need them • There’s no need to install any heavyweight tools like Roswell • Repository size doesn’t grow unboundedly for those that don’t want it • It’s CI-friendly and well-integrated with the UNIX tooling So my setup is: • Make a project/library and write the code interactively using Quicklisp for dependencies. • Once the library is more or less stable dependency-wise, run a 100 LoC dependency-resolving script using Quicklisp metadata. • Run all the commands it suggests (it doesn’t run anything itself, by design!) • Add some leftover deps that Quicklisp doesn’t have or that are special in some other way. (I tend to use Nyxt’s .gitmodules to get dependency locations (seems like working on Nyxt left an imprint, huh?)) • And then set `CL_SOURCE_REGISTRY' to the path with dependencies (usually repository root) when I need to run tests or a custom REPL. Stripped-down project Makefile (yes, I use Make for Lisp projects) then looks like: =================================== make =================================== LISP ?= sbcl LISP_FLAGS ?= ... export CL_SOURCE_REGISTRY ?= $(PWD)// executable: $(LISP) $(LISP_FLAGS) --eval '(require "asdf")' --eval '(asdf:make "system")' --eval '(uiop:quit)' .PHONY: tests tests: $(LISP) $(LISP_FLAGS) --eval '(require "asdf")' --eval '(asdf:test-system "system")' --eval '(uiop:quit)' .PHONY: repl repl: $(LISP) $(LISP_FLAGS) --eval '(require "asdf")' --eval '(asdf:load-system "system")' ==================== Makefile for submodule vendoring approach ==================== Things to note: exported/shared CL_SOURCE_REGISTRY with double slash: That’s a vital hack mentioned by Alexander Artemenko on Reddit . This double slash means a given directory is an ASDF “tree” to be searched for systems recursively. Here’s the syntax explanation in ASDF manual . --eval '(require "asdf")' --eval '(asdf:load-system "system")': All the bootstrapping is basically two expressions, all reliant on ASDF. Given that `CL_SOURCE_REGISTRY' is set up properly, all the dependencies are there. Build, test, and launch REPLs all you want. No Quicklisp needed. repl: REPL is the same call to `$LISP', but without the `--eval '(quit)'' It’s both `rlwrap' compatible and has all the necessary libraries accessible. Swank/Slynk integration: Easy to support by adding two more ASDF subsystems and running them from the Makefile . (A thin strip of an image with lots of boxes, evidently written by an amateur. Some boxes have text on them: “deps”, “CI”, “REPL”, “resilience” (overflowing the box,) and “compat”.) This setup gives me: • Ability to clone the project without dependencies for use with Quicklisp—`git clone project-url' • Ability to clone the project with all the dependencies—`git clone --recursive project-url' • Ability to get only a subset of dependencies I need (say, for libraries not on Quicklisp yet)—`git submodule update --init -- deps/something' or `git submodule update --init -- deps/' to update them all • Global reuse of dependencies with custom ASDF registries (as in .config/common-lisp/source-registry.conf.d/asdf.conf) • Option to run tests from CLI and on CI with all the dependencies accessible to them • Precisely pinned dependencies possibly going fresher than Quicklisp • Custom REPLs in the dev environment • Total Quicklisp / CLPM / OCICL independence, with the only hard project dependency being ASDF (and Quicklisp project repositories at vendoring time, but that’s minor and easy to have locally) • An option to use Quicklisp etc. nonetheless, just do make CL_SOURCE_REGISTRY=path/to/systems// and your dependencies are replacing the submodule ones What not to like about it? ## The Problem With ~/common-lisp (#home-common-lisp) Thanks to Konrad Hinsen for making me think about that! One scenario where this submodule-driven workflow might break is when you clone the repo into `~/common-lisp'. And then there’s another version of one of the deps in there, provoking a version clash. Cloning into `~/common-lisp' is a bad practice that should be eradicated in favor of properly compartmentalized registries. But alas. Anyway, if you clone into `~/common-lisp' or ~/.local/share/common-lisp/source/, then do a non-recursive clone instead. If you cloned the library there, you likely have all the dependencies available already. In the very same `~/common-lisp', no doubt. So no need for submodules. But better not use `~/common-lisp' at all, global state bad. ================================================================================ To see how the whole setup around submodules looks, you can check out my Quickproject template and its makefile and .asd file . CC-BY 4.0 2022-2025 by Artyom Bologov (aartaka). Any and all opinions listed here are my own and not representative of my employers; future, past and present.