Overview
SRFI‑64 is a standardized testing framework for the Scheme programming language, designed to facilitate unit testing and test‑driven development. It provides a consistent API for defining test cases, organizing them into groups, and executing them with configurable runners. The specification includes core procedures for assertions (test‑assert, test‑equal, test‑approximate), hierarchical grouping, and detailed result reporting. Originally written by Per Bothner and finalized in 2005, SRFI‑64 is widely adopted across many Scheme implementations to ensure portable and repeatable testing workflows.
1 Introduction
1.1 Purpose
The primary purpose of SRFI‑64 is to offer a uniform testing interface that works across all conforming Scheme systems. By establishing a standard set of primitives for writing and running tests, it reduces the need for implementation‑specific test harnesses and promotes code portability. The framework supports the test‑driven development (TDD) cycle, allowing developers to write tests before code and then run them automatically.
1.2 Design Rationale
SRFI‑64 was designed with simplicity and extensibility in mind. The core API is intentionally small, consisting of a few fundamental procedures that can be combined to build complex test suites. The design separates test definition from test execution: test specifiers (assertions) are independent of the runner that processes them. This separation allows alternative runners (e.g., graphical, logging, or continuous‑integration‑friendly) to be used without changing the test code.
1.3 Relationship to Other SRFIs
SRFI‑64 is the primary testing SRFI in the Scheme ecosystem. It supersedes earlier ad‑hoc testing practices. Related SRFIs include SRFI‑78 (Lightweight testing) and SRFI‑132 (Sorting libraries, which includes some test utilities), but SRFI‑64 remains the most comprehensive and widely supported standardized testing framework.
2 Specification
2.1 Test Runner Interface
The test runner is the engine that executes test specifiers and reports results. SRFI‑64 defines an abstract runner interface; implementations provide at least one concrete runner.
2.1.1 Default Runner
Every conforming implementation must provide a default runner that prints results to the standard output in a human‑readable format. This runner, created by the procedure (test‑runner) (or equivalent), handles all assertion procedures and group constructs without requiring any user configuration.
2.1.2 User‑Defined Runners
Users may create custom runners by extending the default runner or by implementing the runner interface from scratch. Custom runners can modify output formatting, redirect results to files, or integrate with external tools. The interface is based on a set of hooks (e.g., on‑begin, on‑pass, on‑fail, on‑end) that are called at various points during test execution.
2.2 Test Groups
Test groups allow related test cases to be collected and run together. They also support setup and teardown actions.
2.2.1 Creating Groups
A test group is created using the test‑group macro. Its general form is:
(test‑group <name> <option> ... <body> ...)The <name> is a string identifying the group, and the <body> consists of test specifiers, other groups, and optional setup/teardown forms.
2.2.2 Nesting and Hierarchy
Groups can be nested arbitrarily, forming a tree structure. Each group creates a new scope; test results are aggregated upward so that a parent group’s pass/fail count includes all its descendants. Nesting helps organize large test suites into logical categories.
2.2.3 Group Options
Options may be passed to test‑group to control behavior. Common options include #:setup (a thunk executed before the group’s tests) and #:teardown (a thunk executed after). These are used for resource allocation and cleanup.
2.3 Test Specifiers
Test specifiers are the individual assertions that check whether a condition holds.
2.3.1 test‑assert
(test‑assert <name> <expression>) evaluates <expression> and passes if it returns a true value; otherwise it fails. This is the most basic assertion.
2.3.2 test‑equal
(test‑equal <name> <expected> <actual>) compares <actual> to <expected> using equal?. The test passes only if the two values are equal.
2.3.3 test‑approximate
(test‑approximate <name> <expected> <actual> <error>) checks that <actual> is within <error> of <expected>, using (<= (abs (- <expected> <actual>)) <error>). This is intended for floating‑point comparisons.
2.3.4 test‑error
(test‑error <name> <expression>) expects that evaluating <expression> signals an error. If an error occurs, the test passes; if no error is raised, the test fails.
2.4 Reporting Results
2.4.1 Pass/Fail Counts
After running a test suite, the runner displays the total number of passed and failed assertions. The default runner prints a summary line such as “3 tests, 2 passes, 1 failure”.
2.4.2 Failure Messages
When a test fails, the runner emits a descriptive message that includes the test name, the reason for failure (e.g., the expected and actual values for test‑equal), and the source location if available.
2.4.3 Skipped Tests
The framework supports skipping tests via test‑skip or conditional execution. Skipped tests are reported separately, typically with a count in the summary.
3 Usage Examples
3.1 Basic Test Suite
A minimal test suite using the default runner:
(import (srfi 64))
(test‑begin "Arithmetic")
(test‑assert "positive" (> 5 0))
(test‑equal "addition" (+ 2 3) 5)
(test‑approximate "sqrt" (sqrt 2) 1.414 0.001)
(test‑end "Arithmetic")This prints output like:
%%%% Starting test Arithmetic (Writing full stack trace...)
- positive: PASSED
- addition: PASSED
- sqrt: PASSED
# of expected passes 33.2 Grouping with Setup and Teardown
Using test‑group with options:
(define temp-file #f)
(test‑group "File Operations"
#:setup (lambda () (set! temp-file (make-temp-file)))
#:teardown (lambda () (delete-file temp-file))
(test‑assert "file exists" (file-exists? temp-file))
(test‑assert "file writable" (file-writable? temp-file)))3.3 Custom Reporter
Implementing a simple custom runner that writes results to a file:
(define my-runner (make-test-runner))
(test-runner-on-test-end! my-runner
(lambda (runner)
(let ((result (test-result-kind runner)))
(when (eq? result 'fail)
(display "FAIL: " (current-error-port))
(display (test-runner-test-name runner) (current-error-port))
(newline (current-error-port))))))
(test-runner-reset! my-runner)
(test‑with-runner my-runner (test‑suite ...))4 Implementation Notes
4.1 Conformance Requirements
Implementations must provide all procedures and macros specified in SRFI‑64. They must support at least the default runner and the four assertion types. Optional features (like test‑skip) are recommended but not mandatory. The behavior of the runner when encountering an error inside a test (e.g., a typo in the test expression) is implementation‑defined; robust implementations catch such errors and report them as failures.
4.2 Common Extensions
4.2.1 Colorized Output
Many Scheme implementations extend the default runner to color the output: green for passes, red for failures, yellow for skipped tests. This makes reading test results easier in a terminal.
4.2.2 Parallel Execution
Some implementations provide a parallel runner that executes independent test groups concurrently. This can reduce total test time on multi‑core systems, but care must be taken to avoid races in shared mutable state. The parallel runner is not part of the SRFI‑64 specification but is a common extension.