Lisp (historically stylized as LISP) is a family of programming languages with a long history and a distinctive, fully parenthesized prefix notation. Originally specified in 1958 by John McCarthy, Lisp is the second-oldest high-level programming language after Fortran. It pioneered many concepts in computer science, including tree data structures, automatic storage management, dynamic typing, conditionals, higher-order functions, recursion, the self-hosting compiler, and the read–eval–print loop. The name derives from "LISt Processing," as linked lists are one of the language's major data structures. Lisp dialects, most notably Common Lisp and Scheme, remain in use for artificial intelligence, education, and exploratory programming.
1 History and philosophical foundations
1.1 Origins at MIT (1958–1960)
Lisp was conceived by John McCarthy in 1958 while he was at the Massachusetts Institute of Technology (MIT). McCarthy was motivated by the need for a language suitable for symbolic computation, especially for artificial intelligence research. He published a seminal paper, "Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I" (1960), which laid the theoretical groundwork. The initial implementation ran on an IBM 704 computer and introduced the core concept of S-expressions (symbolic expressions).
1.2 Early dialects and the Lisp 1.5 era
The first widely available dialect was Lisp 1.5, released in 1962. It consolidated early experiments and included a compiler, garbage collection, and a library of list-processing functions. During this period, Lisp spread to other research institutions, notably Stanford University and Carnegie Mellon University, leading to multiple divergent implementations.
1.3 Maclisp, Interlisp, and the 1970s
In the 1970s, two major dialects emerged from the MIT and Stanford traditions. Maclisp, developed at MIT's Project MAC, emphasized performance and became the standard for many AI projects on PDP-10 computers. Interlisp, originating from Bolt, Beranek and Newman (BBN) and later Xerox PARC, pioneered an integrated development environment with a structure editor, spelling corrector, and sophisticated debugging tools. These two dialects influenced each other and set the stage for standardization.
1.4 Standardization: Common Lisp and the ANSI standard
By the late 1970s, the proliferation of incompatible Lisp dialects hindered code sharing and collaboration. In 1981, the ARPA-sponsored Common Lisp initiative aimed to create a unifying dialect. The first specification (Common Lisp: The Language) by Guy L. Steele Jr. appeared in 1984, followed by the ANSI X3J13 standardization effort. The ANSI Common Lisp standard was approved in 1994, incorporating object-oriented features (CLOS) and a comprehensive condition system.
1.5 Scheme: minimalist elegance and influence
Scheme, devised by Gerald J. Sussman and Guy L. Steele Jr. in 1975, was designed as a clean, minimalist dialect emphasizing lexical scoping, proper tail recursion, and first-class continuations. Its small core and formal semantics made it a popular vehicle for teaching computer science concepts. Scheme's influence is seen in many later languages, including JavaScript and Python (through closures and lexical scoping).
1.6 Modern descendants: Clojure, Racket, and others
The early 2000s saw a resurgence of Lisp ideas in new dialects. Clojure, created by Rich Hickey in 2007, runs on the Java Virtual Machine (JVM) and emphasizes immutable data structures and concurrency. Racket (originally PLT Scheme) evolved from Scheme into a platform for language-oriented programming. Other notable descendants include Arc (from Paul Graham) and Hy (a Lisp embedded in Python).
2 Core language features
2.1 S-expressions and the reader
Lisp's syntax is based on S-expressions (symbolic expressions), which are either atoms or cons cells (pairings). The reader is the component that parses textual input into internal data structures. It treats parentheses as structural markers, enabling the uniform representation of code and data.
2.1.1 Atoms, cons cells, and lists
Atoms include symbols (e.g., foo), numbers, strings, and the empty list. Cons cells (pairs) are constructed with the cons function. A list is a chain of cons cells ending with the empty list. For example, (a b c) is syntactic sugar for (cons 'a (cons 'b (cons 'c nil))).
2.1.2 The empty list and the "nil" convention
The empty list is written as () and is also denoted by the symbol nil. In most Lisps, nil is the only false value (aside from explicit ()), while any other value is considered true for conditionals. This dual role of nil as both a symbol and an empty list simplifies many programming patterns.
2.2 Homoiconicity and code-as-data
Lisp is homoiconic: the primary representation of code (S-expressions) is also the primary data structure. This means that a Lisp program can manipulate its own source code as data, enabling powerful metaprogramming. The function eval takes an S-expression and evaluates it as code. This property is foundational for macros and domain-specific languages.
2.3 Garbage collection and memory management
Lisp was one of the first languages to incorporate automatic memory management through garbage collection. Early implementations used a simple mark-and-sweep collector. Modern dialects have evolved sophisticated generational, incremental, and concurrent collectors, making memory management transparent to the programmer.
2.4 Dynamic typing and first-class functions
Lisp is dynamically typed: type checking occurs at runtime, and variables can hold values of any type without explicit declarations. First-class functions allow functions to be passed as arguments, returned from other functions, and stored in data structures.
2.4.1 Lambda functions and closures
Lambdas are anonymous functions created with the lambda special form. A closure is a lambda that captures its lexical environment, preserving access to free variables even after the enclosing scope exits. Closures enable functional programming idioms and are fundamental to Scheme and Common Lisp.
2.4.2 Higher-order functions (map, reduce, etc.)
Higher-order functions operate on other functions. Common examples include map, which applies a function to each element of a list, and reduce (or fold), which combines elements using a binary function. These functions reduce the need for explicit recursion and promote concise, declarative code.
2.5 Macros: compile-time code transformation
Macros are the most distinctive feature of Lisp. They allow programmers to extend the language by defining new syntactic constructs that are expanded at compile time, before evaluation.
2.5.1 Simple macro definition (defmacro)
The defmacro form defines a macro: it takes a name, a parameter list, and a body that returns an S-expression. When the macro is used, the body is executed to generate code, which is then evaluated. For example, a simple when macro can expand to an if form.
2.5.2 Macro hygiene and syntactic abstraction
Hygienic macros (found in Scheme and Racket) prevent accidental variable capture by ensuring that generated code does not interfere with surrounding bindings. Common Lisp's defmacro is unhygienic by default, but libraries such as gensym help manage symbol generation. Syntactic abstraction via macros allows creating custom language constructs, from looping structures to complete DSLs.
2.5.3 Reader macros and custom syntax
Reader macros are functions attached to specific characters (e.g., #, ', ` ) that modify how the reader interprets input. They enable user-defined syntax extensions, such as abbreviated notation for vectors or hash tables. Common Lisp allows defining new reader macros via set-macro-character.
2.6 The interactive environment: REPL and incremental development
The read–eval–print loop (REPL) is a hallmark of Lisp development. Programmers can enter expressions, see immediate results, and modify code on the fly. Combined with dynamic redefinition of functions and classes, the REPL supports interactive, incremental development—a style that is especially productive for exploratory programming and debugging.
3 Major dialects and their ecosystems
3.1 Common Lisp
Common Lisp is a multi-paradigm language supporting procedural, functional, and object-oriented programming. It is standardized by ANSI and has an extensive library ecosystem.
3.1.1 Object system (CLOS)
The Common Lisp Object System (CLOS) provides a powerful and flexible object model. It features multiple inheritance, method combination, and a metaobject protocol (MOP) that allows introspection and customization of the object system itself. CLOS separates methods from classes, using generic functions that dispatch based on all arguments.
3.1.2 Condition system (restarts and handlers)
Common Lisp's condition system is more sophisticated than simple exception handling. Conditions can be signaled, and handlers may choose to proceed by invoking restarts (e.g., retry an operation, use a default value). This separation of signaling from handling provides fine-grained control in complex systems.
3.1.3 The HyperSpec and community
The Common Lisp HyperSpec (CLHS) is an online reference derived from the ANSI standard, maintained by Kent M. Pitman. The community is active through mailing lists, IRC channels (e.g., #lisp on Freenode), and forums like r/lisp. Libraries are distributed via Quicklisp.
3.2 Scheme
Scheme is a minimalist dialect with a strong emphasis on functional programming and formal semantics. Its small core makes it ideal for teaching and language experimentation.
3.2.1 Tail-call optimization and lexical scoping
Scheme requires that tail calls (including recursion) be optimized so that they do not consume stack space, making iterative processes expressible as recursion without memory growth. All variables are lexically scoped by default, with dynamic scoping available only through explicit mechanisms.
3.2.2 Continuations and call/cc
First-class continuations, accessed via call-with-current-continuation (call/cc), allow the program to capture the current execution state as an object that can be invoked later. This enables advanced control flow like coroutines, backtracking, and exception handling.
3.2.3 R5RS, R6RS, R7RS standards
Scheme has evolved through several standard reports: R5RS (1998) is the most widely implemented; R6RS (2007) introduced libraries and a larger library by default; R7RS (2013) provided a smaller standard for embedded systems and a larger "R7RS-large" effort that is ongoing. Implementations like GNU Guile and Chez Scheme remain popular.
3.3 Clojure
Clojure is a modern Lisp dialect designed for concurrent programming and practical interoperability with host platforms.
3.3.1 Hosted on the JVM, CLR, and JavaScript
Clojure runs primarily on the Java Virtual Machine (JVM), directly calling Java libraries. It also targets the Common Language Runtime (ClojureCLR) and JavaScript (ClojureScript), enabling front-end and back-end development from a single codebase.
3.3.2 Immutable data structures and persistent collections
Clojure provides immutable, persistent data structures (e.g., vectors, maps, sets) that efficiently share structure between versions. This eliminates side effects and simplifies concurrent programming.
3.3.3 Concurrency primitives (atoms, refs, agents)
Clojure offers several concurrency models: atoms for synchronous, independent state changes; refs for coordinated, transactional state (via Software Transactional Memory); and agents for asynchronous, independent updates. This design avoids locks and race conditions.
3.4 Emacs Lisp
Emacs Lisp (Elisp) is the scripting language for the GNU Emacs text editor.
3.4.1 Integration with GNU Emacs
Elisp is deeply integrated into Emacs, providing control over buffers, windows, key bindings, and modes. The entire editor is extensible in Elisp, from simple customization to complex IDE features.
3.4.2 Dominance in text editor extensibility
Due to Emacs's long history and enthusiastic user base, Elisp remains one of the most widely used Lisp dialects in terms of active lines of code. It is the primary language for extending Emacs, and its REPL (M-x ielm) allows on-the-fly experimentation.
3.5 Racket
Racket (formerly PLT Scheme) evolved from Scheme into a platform for creating and deploying new programming languages.
3.5.1 Language-oriented programming
Racket's macro system and module system allow programmers to define new languages as libraries. A single Racket program can contain embedded DSLs with their own syntax, editors, and types. This paradigm, called language-oriented programming, is a central design goal.
3.5.2 Teaching and the DrRacket IDE
The DrRacket IDE is designed for educational use, with a stepper, a teaching language hierarchy (Beginning Student, Intermediate, Advanced), and graphical tools for debugging. Racket is widely used in introductory computer science courses, notably at Brown University and the University of Utah.
4 Applications and influence
4.1 Artificial intelligence and expert systems
Lisp was the dominant language for AI research from the 1960s through the 1980s. Early expert systems (e.g., MYCIN, XCON) were written in Maclisp or Interlisp. The language's dynamic nature and symbolic manipulation capabilities made it ideal for knowledge representation and rule-based reasoning.
4.2 Computer science education
Lisp, particularly Scheme, has been used to teach fundamental concepts such as recursion, functional programming, and metacircular evaluation. Seminal textbooks like *Structure and Interpretation of Computer Programs* (SICP) by Harold Abelson and Gerald Jay Sussman have introduced generations of students to Lisp.
4.3 Domain-specific languages and internal DSLs
Thanks to macros and homoiconicity, Lisp excels at building domain-specific languages (DSLs). Examples include the LOOP macro (Common Lisp for iteration), the OPS5 production system, and various embedded query languages. This ability makes Lisp a popular choice for prototyping custom notation.
4.4 Web development and scripting
Several Lisp dialects have been used for server-side web development. Common Lisp frameworks (e.g., Hunchentoot, Caveman2) and Clojure’s Ring/Compojure stack enable HTTP applications. ClojureScript compiles to JavaScript for client-side scripting. Emacs Lisp is used extensively for text processing and scripting within Emacs.
4.5 Notable software written in Lisp (e.g., Emacs, AutoCAD)
Significant pieces of software have been implemented in Lisp or its dialects. GNU Emacs (Elisp) is the most famous. AutoCAD, the CAD software, used a Lisp dialect called AutoLISP for customization. Other examples include the flight booking system Deltamatic (Lisp Machine), the theorem prover ACL2, and the music notation software LilyPond.
5 Tools and development environment
5.1 Compilers and interpreters
Lisp implementations typically offer both a compiler and an interpreter. Common Lisp systems like SBCL (Steel Bank Common Lisp) compile to native machine code, while others like CLISP use bytecode internals. Scheme implementations often provide an interpreter for REPL use and a compiler for efficiency. Clojure compiles to JVM bytecode or JavaScript.
5.2 Software libraries and package managers (Quicklisp, SLIME)
Quicklisp is a library manager for Common Lisp, providing thousands of packages and resolving dependencies automatically. SLIME (Superior Lisp Interaction Mode for Emacs) integrates a Common Lisp REPL with the editor, offering debugging, inspection, and code navigation. For Clojure, Leiningen is the popular build tool and package manager.
5.3 Debugging and profiling
Lisp environments provide powerful debugging tools. Common Lisp's condition system allows interactive restart from errors. SLIME offers a graphical debugger with stack inspection and variable analysis. Profilers (e.g., SBCL's sb-sprof) help identify performance bottlenecks.
5.4 Interactive development with SLIME and Sly
SLIME and its successor Sly (developed for a more modern aesthetic) are Emacs modes that connect to a running Lisp process. They support incremental compilation, macro expansion, and cross-referencing. This tight integration enables a development workflow where the program is constantly evolving without restarting.
6 Criticism and limitations
6.1 Performance concerns and historical overhead
Early Lisp implementations were slow due to interpreted execution and naive garbage collection. Modern Common Lisp compilers (e.g., SBCL) can produce highly optimized code, often competitive with C. However, the dynamic features (type dispatch, macros) can still incur overhead, and some implementations lag in raw speed for scientific computing.
6.2 Readability and parentheses controversy
Lisp's heavy use of parentheses has often been criticized as visually cluttered, leading to jokes about "parenthesis mountain." Many newcomers find the prefix notation and nested parentheses harder to read than infix syntax. Defenders argue that proper indentation and highlighting mitigate this, and that macros allow for cleaner abstractions.
6.3 Fragmentation across dialects
While Common Lisp and Scheme have standardized specifications, the Lisp family remains fragmented. Many dialects (e.g., Emacs Lisp, AutoLISP, Clojure) are incompatible with each other, requiring significant effort to port code. This fragmentation limits library reuse and community cohesion.
6.4 Niche adoption compared to mainstream languages
Despite its historical influence, Lisp has not achieved widespread adoption in industry outside of specific niches (AI, Emacs, some financial systems). The rise of mainstream functional languages (Haskell, Scala, F#) and dynamic languages (Python, JavaScript) has drawn away potential users. However, Clojure has found traction in certain financial and web development circles.
7 Community and culture
7.1 The “Lisp curse” and long-term loyalists
The "Lisp curse" refers to the observation that Lisp programmers often build custom tools and languages so effectively that they lose interest in sharing or promoting them, contributing to the language’s relative obscurity. Despite this, Lisp has a small but intensely loyal community of long-term users who value its expressiveness and power.
7.2 Conferences and user groups (ILC, ECLM)
The International Lisp Conference (ILC) has been held irregularly since 2002, bringing together researchers and practitioners. The European Common Lisp Meeting (ECLM) is a smaller, more focused event. Clojure has its own conferences (Clojure/conj, ClojureD), and local user groups (lisp-user groups) exist in many cities.
7.3 Humor and internet memes (parentheses jokes, Lisp alien)
Lisp culture includes a distinctive sense of humor. Internet memes joke about the abundance of parentheses (e.g., "Lisp programmers don't count parentheses; they just close the right number when the file ends"). The "Lisp alien" (a green, tentacled creature) appears in t-shirts and stickers. These inside jokes reinforce community identity.
8 References and further reading
- McCarthy, John. "Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I." *Communications of the ACM* 3, no. 4 (1960): 184–195.
- Steele, Guy L. Jr. *Common Lisp: The Language*, 2nd ed. Digital Press, 1990.
- Abelson, Harold, Gerald Jay Sussman, and Julie Sussman. *Structure and Interpretation of Computer Programs*, 2nd ed. MIT Press, 1996.
- Graham, Paul. *On Lisp*. Prentice Hall, 1993.
- Queinnec, Christian. *Lisp in Small Pieces*. Cambridge University Press, 1996.
- Seibel, Peter. *Practical Common Lisp*. Apress, 2005.
- Friedman, Daniel P., and Matthias Felleisen. *The Little Schemer*, 4th ed. MIT Press, 1995.
- Hickey, Rich. "Clojure." http://clojure.org.
- *Common Lisp HyperSpec*. http://www.lispworks.com/documentation/HyperSpec/Front/index.htm.
- *Racket Documentation*. http://docs.racket-lang.org.