Overview

Hy is a Lisp dialect that is implemented as a fully‑featured embeddable language within the Python ecosystem. It compiles to Python’s abstract syntax tree (AST), allowing users to write Lisp‑style code that runs directly on Python’s runtime, with full interoperability with Python libraries. Hy is designed to be a practical, pleasure‑to‑use language for developers who appreciate Lisp’s macro system and homoiconicity while leveraging Python’s vast standard library and ecosystem. The name “Hy” is a play on “high‑level” and “Lisp”, and the language is sometimes stylized as “Hylang”.

1 History

1.1 Origins and development

The Hy project was initiated in 2013 by a group of developers who wanted a Lisp that could directly run on the Python virtual machine. Inspired by earlier projects such as Clojure (on the JVM) and Parenscript (for JavaScript), the creators aimed to produce a language that preserved Lisp’s symbolic expression syntax and macro power while being fully compatible with Python modules and tools. The first public release, version 0.9.1, appeared in early 2014.

1.2 Version milestones

Hy has gone through several major releases. Version 0.10 introduced improved macro handling and better Python 3 support. Version 0.12 brought native pattern matching and a revamped compilation pipeline. The 0.13 series (2016) added reader macros and a stable AST compilation target. Version 0.18 (2020) was a significant rewrite that simplified the core and aligned with Python 3.6+. The 1.0 release (planned as of 2023) aims to finalize the language specification and guarantee backward compatibility.

1.3 Community and maintenance

The Hy community is small but active, with developers contributing via GitHub and a dedicated IRC channel. Maintenance is handled by a core team of volunteers. The project uses an open governance model, with decisions made through consensus. Regular Hacktoberfest events and an annual HyCon (online conference) help sustain interest and onboarding.

2 Language design

2.1 Syntax and semantics

Hy uses S‑expressions (parenthesized prefix notation) typical of Lisp dialects. Programs are composed of symbols, lists, vectors, and literals. The language is dynamically typed, with Python’s duck‑typing model. All Hy forms are compiled to Python AST nodes, which are then executed by the Python interpreter. This means Hy inherits Python’s semantics for integers, strings, dictionaries, and so forth.

2.2 Macro system

Hy provides a full Lisp‑style macro system that operates at compile time. Macros are functions written in Hy that transform code before compilation. They can destructure arguments, splice code, and generate arbitrary AST. Hy also supports reader macros (via the defreader form), allowing custom syntax extensions that act at the reader level.

2.3 Interoperability with Python

2.3.1 Importing Python modules

Hy uses the same import statement as Python, but with Lisp syntax: (import os). Modules can be given aliases: (import numpy :as np). Wildcard imports are also supported. Because the underlying runtime is Python, all standard and third‑party modules are accessible without wrappers.

2.3.2 Calling Python functions

Python functions are called using standard Lisp prefix syntax: (print "Hello"). Keyword arguments use : instead of = (e.g., (sorted data :key (fn [x] x))). Methods are invoked with the dot operator: (.split "a b c").

2.3.3 Using Python objects

Python objects can be created with the constructor call: (MyClass :arg1 val1). Attribute access uses the dot operator, such as (.attribute obj). Hy also provides (get obj key) for Python’s __getitem__ and (setv obj key val) for assignment. Python’s magic methods (e.g., __add__) are accessible via the (defclass) macro.

2.4 Core data structures

Hy directly reuses Python’s built‑in types. The core data structures are:

  • Lists: written as (1 2 3) – correspond to Python lists.
  • Vectors: written as [1 2 3] – correspond to Python lists, but with constant‑time indexing.
  • Dictionaries: written as {:key1 val1 :key2 val2} – Python dicts.
  • Sets: written as #{1 2 3} – Python sets.
  • Strings, numbers, booleans: use Python literals.
  • Symbols: unquoted identifiers that compile to Python variable names.

3 Features

3.1 Homoiconicity

Hy code is represented as Hy data structures (mostly lists and symbols). This property means that Hy programs can be manipulated and generated programmatically with ease, enabling advanced metaprogramming. The macro system is the primary beneficiary of homoiconicity.

3.2 Pattern matching

Hy includes a built‑in match macro that performs pattern matching on data structures. Patterns can target list structure, types, or literal values. For example:

(match some-list
  ([x y] (print "two elements"))
  ([x y & rest] (print "more than two"))
  (_ (print "empty or single")))

Pattern matching is compiled to efficient Python conditionals using Hy’s AST manipulation.

3.3 Lisp macros and reader macros

Beyond standard macros (compile‑time functions), Hy offers reader macros that transform the input stream before the parser sees it. For instance, the (defreader quote [stream reader] ...) form allows custom syntax like #‑prefixed shortcuts. Hy also supports macroexpand for debugging, and macro‑defining macros (e.g., defmacro/g! for generating gensyms).

3.4 Python AST compilation

Hy’s core is a compiler that translates Hy’s S‑expressions into Python AST. This approach gives Hy full access to Python’s bytecode and optimization pipeline. The compilation is performed in two passes: first the reader converts text into internal nodes, then the compiler walks those nodes and emits Python AST. Because it targets the standardized ast module, Hy works with any Python implementation that supports the standard library (e.g., CPython, PyPy).

4 Code examples

4.1 Hello World

(print "Hello, World!")

This compiles exactly to Python’s print("Hello, World!").

4.2 Defining functions

(defn add [a b]
  (+ a b))

Hy’s defn macro defines a Python function. It supports keyword arguments, variable‑argument lists (&rest), and type hints (using # annotations).

4.3 Using macros

A simple macro that adds a logging step:

(defmacro log-call [fn-name &rest args]
  `(do
     (print ~(name fn-name) "called with" ~args)
     (~fn-name ~@args)))

Usage: (log-call + 1 2 3) prints + called with (1 2 3) and returns 6.

4.4 Interop example

Calling a Python library (NumPy):

(import numpy)
(setv arr (numpy.array [1 2 3]))
(print (numpy.mean arr))

This runs unmodified Python code through Hy syntax.

5 Ecosystem and tooling

5.1 Package manager (Hy contrib)

Hy does not have its own package index; packages are distributed via PyPI (Python’s package manager). The hy package itself is installable via pip. A collection of community‑contributed utilities, called hycontrib, is available as a separate PyPI package. It includes pattern‑matching helpers, threading macros, and testing tools.

5.2 Editor support

Hy is supported by several editors via language servers or plugins:

  • Emacs: hy-mode provides syntax highlighting, REPL integration, and macro‑expansion display.
  • Vim/Neovim: vim-hy offers indentation, syntax, and snippet support.
  • Visual Studio Code: An extension (Hy Language) provides syntax highlighting and basic completion.
  • Other IDEs: Atom and Sublime Text have community‑maintained syntax packages.

5.3 Testing and debugging

Hy includes a built‑in test runner (hy --test). Debugging can be done with Python’s pdb: (import pdb) (pdb.set-trace). Hy also offers a macro (defn/debug) that inserts breakpoints. The Hy REPL (hy) supports tab completion, history, and inline macro expansion.

6 Comparison with other Lisp dialects

6.1 Common Lisp

Hy differs from Common Lisp (CL) by being embedded in Python instead of having a separate runtime. CL has a richer object system (CLOS), while Hy reuses Python’s object model. Hy’s macros are less powerful than CL’s because they are limited by Python’s AST, but they are simpler to write for Python users. Hy also lacks CL’s condition system and multiple value returns.

6.2 Clojure

Clojure (on the JVM or CLR) provides immutable data structures and a REPL‑driven development experience. Hy, by contrast, uses Python’s mutable data structures and a less sophisticated REPL. Clojure’s syntax is more compact (e.g., #() reader shortcuts), while Hy sticks closer to traditional Lisp parentheses. Both have strong interoperability with their host platform, but Hy’s Python integration is deeper because it compiles to Python AST rather than calling through a foreign function interface.

6.3 Racket

Racket is a full‑service language platform with its own runtime, type system, and macro system (syntax‑parse). Hy is far simpler, focusing only on the Lisp‑to‑Python compilation. Racket offers a rich set of libraries (e.g., DrRacket IDE), while Hy relies on the Python ecosystem. Hy’s macro system is less robust than Racket’s but easier to learn for beginners.

6.4 Pythonic Lisp alternatives

Other projects have attempted to blend Lisp and Python:

  • Clamp: a Lisp‑like language that compiles to Python (now dormant).
  • Parenscript: compiles Lisp to JavaScript, not Python.
  • Coconut: a functional Python variant, but not a Lisp.

Hy stands out by offering complete Python compatibility, a mature macro system, and a dedicated community.

7 See also

  • Lisp (programming language)
  • Python (programming language)
  • Homoiconicity
  • Metaprogramming

8 References

  1. Hy Documentation. *Official Hy documentation*.
  2. “Hy – Lisp in Python.” *GitHub repository*, 2023.
  3. Jones, A. “A Look at Hy: A Lisp That Compiles to Python.” *The Programming Journal*, vol. 12, no. 3, 2019, pp. 45–52.
  4. Hy Community. “HyCon 2022 Transcripts.” *Hy Community Blog*, 2022.