FiveAM is a unit testing framework for the Common Lisp programming language. Named after the early morning hours often associated with debugging sessions, it provides a comprehensive set of macros and utilities for defining, organizing, and executing tests. FiveAM emphasizes simplicity, extensibility, and integration with the Common Lisp condition system, offering features such as test suites, fixtures, test runners, and reporting mechanisms. It is widely used in the Common Lisp community for both small-scale and large project testing.
1.1 Origins and Motivation
FiveAM was created in response to the need for a modern, well-structured testing framework in the Common Lisp ecosystem. Prior to its development, existing tools like RT (Regression Test) and Lisp Unit offered basic functionality but lacked features such as automatic test discovery, fixtures, and extensible reporting. The author, drawing on experience with unit testing in other languages, aimed to design a framework that leveraged Common Lisp’s macro system and condition system to provide a more expressive and flexible testing environment. The name “FiveAM” humorously references the early hour when many debugging sessions extend, nodding to the code’s ability to survive long overnight testing sessions.
1.2 Version History
1.2.1 Initial Release
The first public release of FiveAM appeared in the early 2000s on the open‑source repository Common-Lisp.net. It provided the core macros DEFTEST and DEFSUITE, basic assertion forms, and a simple console test runner. The initial version was well received for its simplicity and low barrier to entry, quickly gaining adoption among Common Lisp projects.
1.2.2 Major Revisions
Subsequent major revisions introduced fixtures, customizable checkers, and multiple output formats. Version 2.0 added fixture scopes and dynamic fixtures, allowing per‑test and per‑suite setup/teardown. Later releases integrated with Quicklisp for easy installation, improved condition handling (capturing unexpected errors as test failures), and added support for Emacs/SLIME integration. The library has remained backward‑compatible through careful semantic versioning, with the current stable release being version 2.1 (as of 2024).
2.1 Test Definition Macros
2.1.1 DEFTEST
DEFTEST is the primary macro for defining an individual test. It takes a test name, an optional list of documentation strings or keywords, and a body of code containing assertions. The macro wraps the body in a condition handler so that unhandled errors are recorded as failures, ensuring clean test isolation. Example:
(deftest test-addition ()
(is (= 4 (+ 2 2))))2.1.2 DEFSUITE
DEFSUITE groups related tests into a named suite. Suites can inherit tests from other suites, enabling hierarchical organization. Tests within a suite can be run together, and suite‑level fixtures (setup/teardown) can be defined. Suites also support :documentation strings and :export options for controlling which tests are visible to the package system.
2.2 Fixtures and Setup/Teardown
2.2.1 Fixture Scopes
FiveAM provides fixtures – code that runs before and after tests – to manage shared state. Fixture scopes control the frequency of execution: :once runs the fixture once per suite run, :each runs it before and after every test in the suite, and :suite runs it at the start and end of the entire suite execution. Scopes are specified in the DEFSUITE form via the :fixture keyword.
2.2.2 Dynamic Fixtures
Dynamic fixtures allow fixture functions to be defined as generic functions and extended using Common Lisp’s CLOS (Common Lisp Object System). By specializing fixture methods on test names or suite classes, users can create context‑dependent setup and teardown logic. This is particularly useful when test database connections or mock objects need to be created per scenario.
2.3 Assertions and Checkers
2.3.1 Basic Assertions
FiveAM includes several assertion macros that test boolean conditions and report failure meaningfully: IS for equality or predicate checks, TRUE and FALSE for general truth value checks, and SIGNALS to assert that a given form signals a specific condition type. Each assertion macro captures the failed form and its values, providing detailed diagnostic output on failure.
2.3.2 Custom Checkers
Users can define custom checkers by creating functions that implement a simple protocol – a predicate function that returns a boolean and an optional error message generator. The DEFCHECKER macro automates this process, registering the checker for use in IS and other assertion forms. This extensibility allows domain‑specific validations (e.g., approximate numeric equality, string matching with wildcards).
2.3.3 Exhaustive Assertion (IS, SIGNALS, etc.)
Beyond the basic forms, FiveAM provides IS as the generic assertion that can take a checker symbol as its first argument. For exhaustive testing, IS can be combined with :fail-on-success to invert the assertion. The SIGNALS macro is specialized to test for expected conditions; it can also capture the signaled condition object for further introspection. Additional macros like FINISHES and NO-SIGNALS round out the assertion set.
2.4 Test Runners and Reporting
2.4.1 Console Runner
The default test runner prints results to the standard output or the current Common Lisp debug I/O. It shows a summary of passed, failed, and errored tests, with detailed failure messages including the source location and the expected versus actual values. Console output can be customized via global variables controlling verbosity and colorization.
2.4.2 Integration with Emacs/SLIME
FiveAM integrates tightly with SLIME (Superior Lisp Interaction Mode for Emacs) through the fiveam-asdf system and the slime-test contrib. Users can run single tests or entire suites directly from an Emacs buffer, with failures highlighted in the source code. The test runner can also populate the Emacs compilation buffer, enabling clickable error navigation.
2.4.3 Output Formats
Alternative output formats can be selected by passing a custom :report-function to the runner. Built‑in formats include TAP (Test Anything Protocol) for integration with CI systems, XML (JUnit‑style) for tools like Jenkins, and a summary‑only mode. Users can write their own output modules using the fiveam-report protocol by subclassing the report class.
3.1 Installation via Quicklisp
FiveAM is available as a Quicklisp distribution. To install, evaluate (ql:quickload :fiveam) in a REPL. After loading, the package :fiveam is available. To verify installation, run (5am:run! (5am:test '5am:test-suite)) – this executes FiveAM’s own test suite.
3.2 Writing a Simple Test
A minimal test file might look like:
(defpackage #:my-package/tests
(:use :cl :fiveam))
(in-package #:my-package/tests)
(deftest test-increment ()
(is (= 2 (1+ 1))))
(run! 'test-increment)The run! function accepts a test name or a suite and executes it, printing results to the console.
3.3 Organizing Test Suites
For larger projects, tests are grouped into suites. A typical structure:
(defsuite :core-suite ())
(in-suite :core-suite)
(deftest test-core () ...)
(defsuite :utils-suite ()
(:depends-on :core-suite))
(in-suite :utils-suite)
(deftest test-utils () ...)Suites can be nested and dependencies declared with :depends-on, ensuring that prerequisite tests pass before dependent suites run.
3.4 Running Tests Programmatically
To run tests as part of a build process, use (run! :suite-name) which returns a fiveam-test-result object. The result can be inspected with functions like passed-tests, failed-tests, and errors. ASDF integration via fiveam-asdf allows tests to be defined in a :test component of the system definition, executed by asdf:test-system.
4.1 Custom Test Runners
FiveAM’s runner architecture is based on the runner class and the runner protocol. Custom runners can control how tests are selected, executed, and reported. A common extension is a parallel runner that executes independent tests in separate threads to reduce overall test time. The runner class has slots for :name, :report, and :result, which can be specialized.
4.2 Custom Assertion Types
Beyond simple checkers, FiveAM allows defining new assertion macros using the DEFINE-ASSERTION macro. This macro takes a name, lambda‑list, and body; the body is expected to call check with a result and an optional description. Custom assertions can then be used in IS forms, allowing domain‑specific testing patterns (e.g., (is (approx= 3.14 pi :delta 0.01))).
4.3 Integration with Other Libraries
4.3.1 Stefil (Steel‑Bank Frame Implementation for Lisp)
FiveAM can be used alongside Stefil, a Common Lisp implementation of the Steel‑Bank Frame (SBF) data model. Integration typically involves writing custom checkers that validate SBF structures against expected schemas, leveraging FiveAM’s assertion reporting for debugging schema mismatches.
4.3.2 Roswell Scripts
FiveAM tests can be embedded in Roswell scripts (a Common Lisp scripting environment) for quick verification without a full ASDF system. Roswell provides a test subcommand that invokes FiveAM on the script’s test forms, enabling easy testing of small utilities and one‑off programs.
5.1 Lisp Unit
Lisp Unit is an earlier framework with a design similar to JUnit. It uses object‑oriented test classes and methods, whereas FiveAM relies on macros and functional composition. FiveAM’s macro‑based approach makes test definition more concise, while Lisp Unit provides a more familiar class‑based API for developers coming from Java or C++. FiveAM also has better support for fixtures and custom reporters.
5.2 RT (Regression Test)
RT is a minimal testing library that uses a simple deftest macro and global result lists. It lacks suites, fixtures, and extensible reporting. FiveAM builds on RT’s simplicity by adding organizational and reporting features without forcing a particular project structure. For very small projects, RT may be sufficient, but FiveAM is preferred for maintainable test suites.
5.3 Prove
Prove is a newer framework inspired by Perl’s Test::More. It provides a fluent assertion syntax and integrated test counting. FiveAM is more mature and has a larger community base. Prove’s strength is its typing‑friendly interactive experience, while FiveAM offers deeper integration with the condition system and broader external tool support (e.g., Emacs, TAP). Both frameworks are actively maintained, and choice often depends on personal preference.
6.1 Common Lisp Software Development
- ASDF (Another System Definition Facility)
- Quicklisp
- SLIME (Superior Lisp Interaction Mode for Emacs)
- Common Lisp HyperSpec
6.2 Unit Testing Methodologies
- Test‑Driven Development (TDD)
- Behavior‑Driven Development (BDD)
- Continuous Integration (CI) pipelines
- Code coverage analysis tools