The LOOP macro is a linguistic construct in the Common Lisp programming language that provides a concise, expressive, and versatile way to perform iterative computation. Originating from early Lisp dialects and later standardized in Common Lisp (ANSI X3.226:1994), the LOOP macro combines iterative control structures (e.g., FOR, WHILE, UNTIL) with data collection, accumulation, and conditional execution into a single, declarative‑style syntax. It is often described as a "minilanguage" embedded within Lisp, enabling both simple counting loops and complex data‑driven iterations over lists, vectors, hash tables, and streams. Its power and flexibility make it a staple in Lisp code, though its syntax can be unfamiliar to programmers accustomed to traditional loop constructs.
1 Overview
1.1 History and Standardization
1.1.1 Origins in Interlisp and Maclisp
The LOOP macro traces its origins to the early Lisp dialects Interlisp and Maclisp in the 1970s. Interlisp introduced an early form of a loop facility that allowed iterative constructs with a syntax distinct from traditional Lisp S-expressions. Maclisp also experimented with loop-like macros, though neither dialect had a fully standardized version. These early implementations provided the conceptual foundation for a more unified loop construct, emphasizing readability and the ability to express common iteration patterns without requiring explicit recursion or lower‑level control flow.
1.1.2 Adoption in Common Lisp
During the design of Common Lisp in the early 1980s, the LOOP macro was proposed as part of the language specification. It underwent several revisions, with input from the Lisp community, and was ultimately included in the ANSI Common Lisp standard (ANSI X3.226:1994). The standardization process resolved many syntactic ambiguities and ensured portability across implementations. Since then, LOOP has become a core feature of Common Lisp, widely used in both production and educational code.
1.2 Core Syntax and Semantics
The LOOP macro begins with the symbol loop followed by a series of clauses. The macro processes these clauses to generate an iterative construct with an implicit loop body. The general form is:
(loop clause*)Each clause modifies the behavior of the loop: variable clauses introduce iteration variables, termination clauses control when the loop ends, action clauses execute code, and accumulation clauses gather results.
1.2.1 Loop Clause Types
1.2.1.1 Variable Clauses (FOR, WITH, AS)
- FOR and AS (synonyms) introduce iteration variables that are updated each loop iteration. They can iterate over ranges, sequences, lists, hash tables, or custom generators. Example:
(loop for i from 1 to 10 ...) - WITH introduces local variables that are initialized once before the loop begins and can be updated manually. Example:
(loop with sum = 0 for i in list do (incf sum i))
1.2.1.2 Termination Clauses (WHILE, UNTIL, REPEAT)
- WHILE condition: the loop continues as long as the condition is true.
- UNTIL condition: the loop continues until the condition becomes true.
- REPEAT n: executes the loop body exactly n times.
These clauses can be combined, and the loop exits when any termination condition is met.
1.2.1.3 Action Clauses (DO, RETURN)
- DO body-forms: evaluates the given forms for side effects each iteration.
- RETURN expression: immediately exits the loop, returning the value of expression.
Action clauses allow imperative code within the loop.
1.2.1.4 Accumulation Clauses (COLLECT, APPEND, NCONC, SUM, COUNT, MAXIMIZE, MINIMIZE)
These clauses automatically gather values from the loop body into a result. For example:
(loop for x in list collect (* x 2))returns a list of doubled values.(loop for i from 1 to 100 sum i)returns the sum of numbers 1 to 100.
Each clause has its own semantics for how values are combined (e.g., APPEND concatenates lists, NCONC destructively modifies, MAXIMIZE keeps the maximum).
1.2.2 Loop Body and Implicit Progression
The loop body consists of the clauses that do not specify explicit update rules. The macro automatically updates iteration variables (e.g., incrementing a FOR variable or moving to the next element of a list) at the end of each iteration. The programmer does not need to write explicit increment or update code; the LOOP macro handles progression based on the iteration specification.
1.3 Examples of Common Usage
1.3.1 Simple Numeric Iteration
(loop for i from 1 to 5 do (print i))
;; prints: 1 2 3 4 51.3.2 Iterating Over Lists and Sequences
(loop for x in '(a b c) collect (string-upcase x))
;; => ("A" "B" "C")1.3.3 Iterating Over Hash Tables
(let ((ht (make-hash-table)))
(setf (gethash 'a ht) 1 (gethash 'b ht) 2)
(loop for key being the hash-keys of ht collect key))
;; => (A B) (order unspecified)1.3.4 Nested Loops and Combining Clauses
(loop for x from 1 to 3
collect (loop for y from 1 to x collect y))
;; => ((1) (1 2) (1 2 3))2 Advanced Features
2.1 Loop Iteration Paths and Destructuring
2.1.1 Iterating Over Multiple Variables
Multiple FOR clauses can operate independently or in parallel. By default they iterate simultaneously (parallel), each step updating all variables once. Example: (loop for x in list for y from 0 do ...) updates x and y in lockstep.
2.1.2 Destructuring Loop Variables
Using FOR (var1 var2) in list-of-pairs, LOOP supports destructuring binding, extracting elements from nested structures. This works similarly to DESTRUCTURING-BIND. For example:
(loop for (name age) in '((Alice 30) (Bob 25)) collect name)
;; => (ALICE BOB)2.2 Conditional Clauses (IF, WHEN, UNLESS)
2.2.1 Combining Conditionals with Accumulation
Conditional clauses can be placed inside the loop body, but LOOP also provides a conditional syntax for accumulation clauses: (loop for x in list when (oddp x) collect x). The WHEN or UNLESS modifies only the immediately following clause.
2.2.2 The ELSE Clause
LOOP’s IF clause supports an ELSE branch, allowing different actions for true/false conditions within a single iteration. Example: (loop for x in list if (oddp x) collect x else collect (- x)).
2.3 Data Collection and Accumulation
2.3.1 COLLECT / APPEND / NCONC
- COLLECT gathers values into a new list.
- APPEND concatenates sublists into a single list.
- NCONC destructively modifies sublists to form the result, potentially more efficient for large lists.
2.3.2 SUM / COUNT / MAXIMIZE / MINIMIZE
- SUM adds numeric values.
- COUNT counts non‑nil values or occurrences matching a condition.
- MAXIMIZE and MINIMIZE track the maximum or minimum value encountered.
2.4 Loop Control and Efficiency
2.4.1 Named Loops and Early Exit (FINALLY, RETURN, LOOP‑FINISH)
A loop can be named with (loop named my-loop ...) to allow early exit via (return-from my-loop value). The RETURN clause or the function LOOP‑FINISH also provide early exit. FINALLY clauses execute after the loop terminates naturally.
2.4.2 Loop Prologue and Epilogue (INITIALLY, FINALLY)
- INITIALLY forms execute once before the first iteration.
- FINALLY forms execute once after the last iteration, regardless of how the loop ended (unless an early return bypasses them).
2.4.3 Avoiding Side Effects with Purely Functional Loops
By combining accumulation clauses and avoiding DO clauses, LOOP can be used in a functional style. For example, (loop for i from 1 to 10 collect i) is side‑effect free. However, the loop itself is still imperative under the hood.
3 Comparison with Other Iteration Constructs
3.1 Traditional Lisp Recursion
Recursion is a natural idiom in Lisp, but it can be less intuitive for iteration over many elements and may be limited by stack depth. LOOP offers a more concise and stack‑safe alternative, especially for simple accumulations.
3.2 DO and DOTIMES Macros
DO and DOTIMES provide a more traditional imperative loop with explicit bindings and step forms. LOOP is more declarative and often shorter, but DO is more flexible for low‑level control. Many Lispers prefer LOOP for clarity in data‑driven tasks.
3.3 MAP, MAPCAR, and Series
MAP and MAPCAR apply a function to elements and return a list. They are purely functional. LOOP can mimic this behavior with COLLECT, but also supports side effects and complex termination. The Series library (a separate Lisp library) offers a stream‑based approach similar to LOOP but with lazy evaluation. LOOP is part of the standard; Series is not.
3.4 Contrast with Imperative Languages (C, Python, Java)
In C, loops use explicit initialization, condition, and increment (e.g., for(i=0;i<n;i++)). Python’s for x in list is somewhat similar but lacks built‑in accumulation syntax. Java’s enhanced for loop is also limited. LOOP’s minilanguage approach is far more expressive for combining iteration, filtering, and accumulation into a single construct, though it may be less readable to programmers unfamiliar with its syntax.
4 Implementation and Internals
4.1 Macro Expansion
4.1.1 Transformation to Tagbody and Go
The LOOP macro expands into a lower‑level form using TAGBODY and GO. Each clause is translated into a series of labels, jumps, and variable updates. This expansion preserves the semantics of the original LOOP expression while allowing the compiler to generate efficient code.
4.1.2 Efficiency Considerations
Because the expansion is done at compile time, there is no runtime overhead from macro processing. The resulting code is typically as efficient as hand‑written DO loops, as long as the compiler performs standard optimizations. Some implementations may inline the expansion entirely.
4.2 Compiler Optimizations
Common Lisp compilers often recognize common patterns in LOOP expansion and apply optimizations such as constant folding, strength reduction, and unrolling. The destructiveness of NCONC clauses can be optimized to avoid unnecessary copying. Inline type declarations (e.g., (loop for i of-type fixnum from 1 to 100)) further assist optimization.
4.3 Extensibility: Custom Loop Clauses (DEFMACRO, DEFINE‑LOOP‑CLAUSE)
4.3.1 Defining Simple Loop Clauses
Programmers can extend the LOOP macro by using DEFINE‑LOOP‑CLAUSE (provided by some implementations) or by writing a wrapper macro that expands to a standard LOOP. For example, one could define a DISTINCT clause that collects unique values.
4.3.2 Advanced: Parsing and Code Generation
The LOOP macro’s internal parser is not standardized, so adding entirely new clause types is implementation‑dependent. Some Lisp systems allow hooking into the expansion process to recognize new keywords and generate appropriate expansions. This is rarely needed in practice but demonstrates the macro’s flexibility.
5 Common Pitfalls and Best Practices
5.1 Readability vs. Brevity
While LOOP can express a lot in few characters, overly dense loops become unreadable. It is best to keep loops simple; break complex logic into helper functions or use separate LOOP forms. Consistent formatting (e.g., indentation of clauses) improves clarity.
5.2 Misunderstanding Scope and Variable Binding
Variables introduced by FOR, WITH, and other clauses have specific scoping rules. For example, variables in a FOR clause are rebound each iteration, while WITH variables persist across iterations. Confusion can lead to unexpected behavior. Using (loop with x = 0 for i in list ... (setf x (+ x i))) is different from (loop for i in list sum i).
5.3 Overly Complex Expressions
Nesting many clauses or using multiple accumulation types in one loop can obscure the intent. When the loop becomes too long, consider using a separate function or splitting into multiple loops.
5.4 Performance Traps (e.g., Repeated Evaluation)
In clauses like (loop for i from 0 to (length list) ...), the expression (length list) is evaluated each iteration unless the loop is optimized. Best practice is to compute such values outside the loop or use (loop repeat (length list) ...). Also, destructively modifying lists during iteration can cause undefined behavior.
6 Cultural and Pedagogical Significance
6.1 The Loop Macro in Lisp Education
LOOP is often taught early in Common Lisp courses because it provides a gentle introduction to iteration without requiring understanding of recursion or macro expansion. Its resemblance to natural language (e.g., for i from 1 to 10 collect i) helps beginners grasp the concept quickly.
6.2 Community Debates (Procedural vs. Functional Style)
The Lisp community has a long‑standing debate over the use of LOOP versus more functional constructs (e.g., MAP, REDUCE, recursion). Proponents of functional style argue that LOOP encourages mutable state and side effects, while supporters of LOOP praise its expressiveness and readability for data‑processing tasks. This debate is part of a larger discussion about programming paradigms within Lisp.
6.3 Influence on Other Languages (e.g., Clojure's For, Python Comprehensions)
LOOP’s influence can be seen in Clojure’s for macro, which provides a similar list‑comprehension‑like syntax. Python’s list comprehensions and generator expressions also echo the combination of iteration and accumulation in a declarative style, though Python’s syntax is more limited. LOOP remains a unique feature of Common Lisp, admired for its power and sometimes criticized for its complexity.