Modern, functional Common Lisp: Myths and best practices

Common Lisp has earned the reputation of being difficult to set up. Today, some good solutions exist to help newcomers:

• Portacle is an all-in-one bundle that includes SBCL , Emacs , SLIME and more. While the user must learn the basics of Emacs, Portacle is known to be relatively friction-less.

• The Guix package manager can set up a Common Lisp development environment in one go:

  guix install sbcl git emacs emacs-slime emacs-helm-slime emacs-company emacs-magit emacs-paredit emacs-rainbow-delimiters

  Replace guix install with guix environment --ad-hoc if you don’t want to persist a profile.

  If you prefer VI-style bindings:

  guix install sbcl git emacs emacs-slime emacs-helm-slime emacs-company emacs-magit emacs-paredit emacs-rainbow-delimiters emacs-evil emacs-evil-collection

  If you prefer Lispy over paredit to automate s-expression manipulation (such as automatically balancing parentheses):

  guix install sbcl git emacs emacs-slime emacs-helm-slime emacs-company emacs-magit emacs-lispy emacs-rainbow-delimiters

  Should you need Common Lisp libraries, you can also install many of them with Guix. For the rest, you’ll have to resort to Quicklisp which is not provided by Guix, so you’ll have to install it manually: https://www.quicklisp.org/beta/

The community-maintained Common Lisp cookbook is probably the best starting point with excellent references to everything from string manipulation to continuous integration.

It’s also a good portal to other resources such as:

• awesome-cl : A curated list of Common Lisp frameworks and libraries.
• Practical Common Lisp : A free online book to learn Common Lisp.

That said, the introductory documentation can be rather overwhelming in my opinion and I’ve invited the community to work on a gentle, modern primer for the complete beginner:

https://github.com/LispCookbook/cl-cookbook/issues/270

Feedback welcome!

Many Common Lisp developers use Emacs. You don’t have to, many other editors support Common Lisp as well.

SLIME and SLY are famous for providing a stellar interactive development experience.

On top of this, there are Helm packages to enhance your SLIME / SLY experience with live, narrowing fuzzy-search and completion for about everything:

• https://github.com/emacs-helm/helm-slime
• https://github.com/emacs-helm/helm-sly

SLY is a fork of SLIME. SLIME tends to be more conservative while SLY is slightly more featureful.

A few benefits of SLY over SLIME:

• It has more consistent, cleaner code and reuses more of Emacs facilities (like comint-mode for the REPL).
• It has a better color theme in my opinion.
• It offers better fuzzy completion.
• A feature unique to SLY: stickers allow the user to “stick” non-persistent code to help with debugging by annotating it. See https://github.com/joaotavora/sly#stickers .
• The setup is essentially zero-config with sane defaults, with all the features like mrepl, stickers, etc. enabled by default. Of course you can disable the features you don’t like.
• It compiles with ASDF which makes it easier to package, bundle or deploy.

SLIME does include some features that are only available in SLY as plugins, such as ASDF management. I recommend the following plugins for SLY:

• sly-asdf
• sly-quicklisp
• sly-macrostep

Check out the following not-so-well-known commands, they could dramatically improve your workflow! (Most of them have a SLY equivalent.)

• slime-who-*
• slime-eval-last-expression-in-repl ( C-c C-j )
• slime-list-compiler-notes
• slime-export-symbol-at-point ( C-c x )
• slime-export-class , slime-export-structure
• slime-trace-dialog-toggle-trace ( C-c M-t )
• slime-inspect-definition
• slime-delete-system-fasls (Useful when .fasls are out-of-sync)
• slime-repl-clear-buffer ( C-c M-o : useful when lispy or paredit goes berserk)
• slime-profile-package , then run the desired functions, then slime-profile-report .
• hyperspec-lookup-format and hyperspec-lookup-reader-macro .

In particular, note that slime-who-specializes lists the methods of a given class, which answers a common complaint coming from people used to languages from the Algol family: the ability to complete the methods of the foo class by typing foo.<TAB> .

Historically ASDF required you to explicitly list the files to compile. This was cumbersome and error-prone, but thankfully it has been fixed some time ago with the package-inferred-system extension which I highly recommend:

https://common-lisp.net/project/asdf/asdf/The-package_002dinferred_002dsystem-extension.html#The-package_002dinferred_002dsystem-extension

Benefits:

• No need to list files.
• No need to list dependencies. With non-inferred-systems, the dependency list can be wrong, e.g. include too many of them or miss some. In that sense inferred-systems are more robust.
• Enforces that every file must be a package.

Trick:

• If your system has many files, and thus that many packages, it can get quickly cumbersome for the user to load all those packages one by one. Instead, you can create a “meta package” which imports all the other packages of your system, so that the user only has to load this one to access the whole system.

• How can we get the list of dependencies since we don’t list them explicitly in the .asd anymore? Thankfully the snippet below will retrieve them programmatically for us:

  (declaim (ftype (function (string)) package-dependencies))
  (defun package-dependencies (pkg-name)
    (let (depends)
      (labels ((iter (openlist)
  	       (if (null openlist) depends
  		   ;; is this a subsystem of foo?
  		   (let ((find (search  pkg-name (first openlist))))
  		     (if (and find (zerop find))
  			 (iter (append (asdf:system-depends-on (asdf:find-system (first openlist))) (rest openlist)))
  			 ;; if not, it's a direct dependency: collect it
  			 (progn
  			  (pushnew (first openlist) depends :test 'equalp)
  			  (iter (rest openlist))))))))
        (iter (list pkg-name)))))

Then:

(package-dependencies "dbus")
; => ("ironclad cl-xmlspam split-sequence flexi-streams ieee-floats iolib babel trivial-garbage alexandria

The following packages are widespread and useful enough that can be use -d without hesitation:

• Alexandria : various language and functional programming procedures.
• UIOP: Macros and operating-system-level procedures. Included with ASDF in most Common Lisp compilers.
• trivia : Pattern matching.

The following libraries are also extremely useful, although not to everyone’s taste:

• Trivial-types : More common type definitions.
• Serapeum : a huge collection of utilities (named let, local definitions, hooks, file “human” size, string manipulation, etc.), built as a complement to Alexandria and UIOP. It greatly extends the language.
• Series : Lazy sequences and supporting higher-order functions.

Other language-enhancing libraries:

• Rove : The (only?) testing framework with coverage support.
• ppcre : Advanced string manipulation with unicode support.
• named-readtables : First class name space for readtables.
• generic-cl : Generic function wrapper over various functions in the Common Lisp standard, such as equality predicates and sequence operations.

From the HyperSpec:

(defun equidimensional (a)
  (or (< (array-rank a) 2)
      (apply #'= (array-dimensions a)))) ; =>  EQUIDIMENSIONAL
(deftype square-matrix (&optional type size)
  `(and (array ,type (,size ,size))
	(satisfies equidimensional))) ; =>  SQUARE-MATRIX

Some useful types from the trivial-types package:

(deftype pathname-designator ()
  '(or string
       file-associated-stream
       pathname))

(deftype function-designator ()
  '(or symbol
       function))

(deftype string-designator ()
  '(or character
       symbol
       string))

In practice:

(declaim (ftype (function (trivial-types:pathname-designator) pathname) my-basename))
(defun my-basename (path)
  (uiop:pathname-parent-directory-pathname path))

(compile nil '(lambda () (my-basename 17)))
; in: LAMBDA ()
;     (MY-BASENAME 17)
;
; note: deleting unreachable code
;
; caught WARNING:
;   Constant
;     17 conflicts with its asserted type
;     (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING
;         (AND STREAM (SATISFIES TRIVIAL-TYPES:FILE-ASSOCIATED-STREAM-P)) PATHNAME).
;   See also:
;     The SBCL Manual, Node "Handling of Types"
;
; compilation unit finished
;   caught 1 WARNING condition
;   printed 1 note

To increase the level of warning detection, call

(declaim (optimize (speed 0) (space 0) (debug 3)))

before loading a system.

Also reload -ing a system might raise more warnings. You can use ,reload-system in SLIME / SLY.